> I would be interested in others' comments though, because I once > wanted to maintain a rolling buffer like this for audio data (i.e. a > much faster data rate) and I didn't find any simple solutions on > offer.
I didn't look too close at the sample data, but it is possible to use random access and do this without copying the contents of the file. If the data collected over a day is fixed size or bounded, lets say that it fits in 1MB, then you right up front make your file contain 30 1MB chunks. To read in a given day, you compute where in the file it will be, then read 1MB of info, which could be one record, N elements of an array, a flattened string, or N lines in a text file, just as long as you know what it takes to read the day's 1MB of info. To overwrite a day, you seek to the same spot and do a write that corresponds to the read's parameters. If you can't compute the position in the file without additional info, you place a header record either at the front or end of the file and put the info you need in there, even a full table of contents stating which date is in which record. To use the header, you first read the header record, then compute the location (adding the header size to the position), then do the read or write like before. And if the data isn't fixed or bounded, you can do this more generally by putting more info in the table of contents such as an array of record locations and putting more than 30 records in the file. When a day is overwritten, you first overwrite the records belonging to that day, then append to the end of the file. If the new record is smaller than the old one, you add the records to the Available list in the table of contents. If the table of contents is no longer fixed size, you store it at the end of the file and possibly relocate it each time the file is updated. Anyway, my point is that file I/O can be very efficient with the right approach, but it takes a bit of thought and some programming to achieve it, which is why with small files your suggestion is right on the money. But when the files get large enough, play with the seek and offset fields on the file I/O blocks and you can make almost anything out of them. Greg McKaskle
