There is one big downside though. My Two keys are field_name and symbol_name. When I add tables/groups for field_name/symbol_name, I can walkgroups and quickly found out what the unique field_names and symbol_names are in the file. However, if I have 10 million plus rows+, is there a sql equivalent of 'distinct' which will give me what the unique field_names and symbol_names are in the table (assuming that both field_names and symbol_names are indexed columns) without having to load all the rows?
-----Original Message----- From: Francesc Alted [mailto:fal...@pytables.org] Sent: Wednesday, September 16, 2009 11:10 AM To: pytables-users@lists.sourceforge.net Subject: Re: [Pytables-users] Is it better to have many smaller tables or one large table A Wednesday 16 September 2009 15:18:04 Vineet Jain escriguĂ©: > I have a table with two fields: date (str 8) and value (float32). > > > > I created two files and am trying to explain the file size difference > between the two: > > > > File 1: > > 3 groups and 2 tables. Each table has 390 rows. Total number of rows: 780 > > > > File 2: > > 3 groups and 102 tables. Each table has 7 rows. Total number of rows: 784 > > > > File 2 is 1.3MB while file 1 is 43k. In my design I was going to have > 1000's of tables and hundred's of groups. Given the size difference between > the two files, is it better to have few table with extra keys or a large > number of smaller tables? The difference is due to the fact that file 1 has to put much more metadata (i.e. data that describes data) in there, while in file 2 metadata is minimal. File 1 structure is always preferred over file 2 because it is more scalable. Hope that helps, -- Francesc Alted ---------------------------------------------------------------------------- -- Come build with us! The BlackBerry® Developer Conference in SF, CA is the only developer event you need to attend this year. Jumpstart your developing skills, take BlackBerry mobile applications to market and stay ahead of the curve. Join us from November 9-12, 2009. Register now! http://p.sf.net/sfu/devconf _______________________________________________ Pytables-users mailing list Pytables-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/pytables-users ------------------------------------------------------------------------------ Come build with us! The BlackBerry® Developer Conference in SF, CA is the only developer event you need to attend this year. Jumpstart your developing skills, take BlackBerry mobile applications to market and stay ahead of the curve. Join us from November 9-12, 2009. Register now! http://p.sf.net/sfu/devconf _______________________________________________ Pytables-users mailing list Pytables-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/pytables-users
