"Flemming Frandsen" <[EMAIL PROTECTED]> writes: > "Jason E. Stewart" wrote: > > > True. I just wanted to have a way to do it in C so that it would be > > fast. I want to access arrays of 6 floats in tables with a million > > rows, doing that using perl's split() and join() is going to be slow. > > Why use arrays at all? > > Why not normalize your data (put the n floats in another table) or add > the 6 floats to the rows you are dealing with?
Hey Flemming, I gave a ridiculously simple example just as an idea. I would *love* to be able to normalize my data, but I'm having real trouble figuring it out. Here's the problem I'm trying to solve. I'm building a tool that will enable scientists to load there experimental data into a database. That data will come as spreadsheets of data from scientists. Each group of scientists will use slightly different technology to generate the data so those spreadsheets are very likely to have different numbers of columns, and they will certainly have different data types in the various columns, however from one group, all the data should have the same format (or small set of formats). So whatever solution I come up with, needs to be flexible and store data no matter how many columns it has or what the data types for the fields are. One complication is that there are likely to be millions of rows from the spreadsheets, so I want it to be reasonable efficient (no joins if possible). Variable length arrays seemed the obvious way to solve this. I just wanted to avoid having to create a new table for each spreadsheet configuration. A small finite number of tables would be fine, but I couldn't come up with a way. If you have other ideas, I'd love to hear them. > If you insist on having arrays in fields then take a look at Storable, > it's C code and quite fast and it will serialize/unserialize data > structures for you. Thanks. I will. jas.
