Some background...
We have Objects r1, r2 ... r1600 Which each have (~1600) regions d1, d2 ... d500 Which each have (~500) datapoints So, roughly we have around 786,000 datapoints on a given object. Typically, we might take 15 or so regions and take data from each of their datapoints (~7500) at ~1300 "steps" or environmental variations. So, each test will generate about 9.75M floats. A curve can be extracted from the 1300 steps (floats) for each datapoint. We then run these tests hundreds of times a day - sometimes on different objects, sometimes on the same ojbects (so you can analyse the curves through time) We have to date stored this raw data within compressed text files indexed by MySQL. However, as you can imagine, querying this data is a pain. We have to ask MySQL for all of the tests of a given object, then analyse the files to extract the appropriate curves - often opening up 100s of datafiles. The question is whether anyone has any more intelligent ways of storing this data within MySQL without busting MySQLs file size limits, or reasonable CPU/RAM contraints. Storing the data raw in rows doesn't seem like the most sane technique. Nor does throwing it into a char for each datapoint. What I'm looking for is a _lossless_ technique to store the curves within the database for each datapoint (we need to be able to extract and analyse data for each datapoint). My guess is that someone out there is doing scientific analysis similar to this using MySQL as their backend. The environment is R&D so queries will be fairly limited. We obviously don't want queries to take down the DB server, but, at the same time this isn't the backend data to Yahoo! serving millions of requests every minute. Thanks! Seth --------------------------------------------------------------------- Before posting, please check: http://www.mysql.com/manual.php (the manual) http://lists.mysql.com/ (the list archive) To request this thread, e-mail <[EMAIL PROTECTED]> To unsubscribe, e-mail <[EMAIL PROTECTED]> Trouble unsubscribing? Try: http://lists.mysql.com/php/unsubscribe.php
