Hello Jim,
Tuesday, May 11, 2010, 4:46:46 PM, you wrote: JF> On Tue, May 11, 2010 at 7:47 AM, Adam GROSZER <agros...@gmail.com> wrote: >> Hello Jim, >> >> Tuesday, May 11, 2010, 1:37:19 PM, you wrote: >> >> JF> On Tue, May 11, 2010 at 7:13 AM, Adam GROSZER <agros...@gmail.com> wrote: >>>> Hello Jim, >>>> >>>> Tuesday, May 11, 2010, 12:33:04 PM, you wrote: >>>> >>>> JF> On Tue, May 11, 2010 at 3:16 AM, Adam GROSZER <agros...@gmail.com> >>>> wrote: >>>>>> Hello Jim, >>>>>> >>>>>> Monday, May 10, 2010, 1:27:00 PM, you wrote: >>>>>> >>>>>> JF> On Sun, May 9, 2010 at 4:59 PM, Roel Bruggink <r...@fourdigits.nl> >>>>>> wrote: >>>>>>>> That's really interesting! Did you notice any issues performance wise, >>>>>>>> or >>>>>>>> didn't you check that yet? >>>>>> >>>>>> JF> I didn't check performance. I just iterated over a file storage file, >>>>>> JF> checking compressed and uncompressed pickle sizes. >>>>>> >>>>>> I'd say some checksum is then also needed to detect bit failures that >>>>>> mess up the compressed data. >>>> >>>> JF> Why? >>>> >>>> I think the gzip algo compresses to a bit-stream, where even one bit >>>> has an error the rest of the uncompressed data might be a total mess. >>>> If that one bit is relatively early in the stream it's fatal. >>>> Salvaging the data is not a joy either. >>>> I know at this level we should expect that the OS and any underlying >>>> infrastructure should provide error-free data or fail. >>>> Tho I've seen some magic situations where the file copied without >>>> error through a network, but at the end CRC check failed on it :-O >> >> JF> How would a checksum help? All it would do is tell you your hosed. >> JF> It wouldn't make you any less hosed. >> >> Yes, but I would know why it's hosed. JF> How so? How would you know why it is hosed. Because of data corruption in the compressed stream. JF> Note BTW that the zlib format already includes a checksum. JF> http://www.faqs.org/rfcs/rfc1950.html I missed that. Case closed then ;-) Sorry for the noise. -- Best regards, Adam GROSZER mailto:agros...@gmail.com -- Quote of the day: Some are atheists by neglect; others are so by affectation; they that think there is no God at some times do not think so at all times. - Benjamin Whichcote _______________________________________________ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
