On Thu, Dec 4, 2008 at 17:43, Brennan Williams <[EMAIL PROTECTED]> wrote: > [EMAIL PROTECTED] wrote: >> On Thu, Dec 4, 2008 at 6:17 PM, Brennan Williams >> <[EMAIL PROTECTED]> wrote: >> >>> My app reads in one or more float arrays from a binary file. >>> >>> Sometimes due to network timeouts etc the array is not read correctly. >>> >>> What would be the best way of checking the validity of the data? >>> >>> Would some sort of checksum approach be a good idea? >>> Would that work with an array of floating point values? >>> Or are checksums more for int,byte,string type data? >>> >>> >> >> If you want to verify the file itself, then python provides several >> more or less secure checksums, my experience was that zlib.crc32 was >> pretty fast on moderate file sizes. crc32 is common inside archive >> files and for binary newsgroups. If you have large files transported >> over the network, e.g. GB size, I would work with par2 repair files, >> which verifies and repairs at the same time. >> >> > The file has multiple arrays stored in it. > > So I want to have some sort of validity check on just the array that I'm > reading.
So do it on the bytes of the individual arrays. Just don't bother implementing new type-specific checksums. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco _______________________________________________ Numpy-discussion mailing list [email protected] http://projects.scipy.org/mailman/listinfo/numpy-discussion
