OK...I know I'm talking to myself here, but it's worth documenting the thoughts in case they are useful...
*If *pyfuse returned a valid representation of nano-seconds since 1970 (say, for example, a python3 INT value -- which is unbounded), then S3QL would need to store them differently, since SQLite will do nothing more than 64 bit numeric values. There are three broad possibilities, I think: - store as strings. This would be a resounding 'no' since comparisons would not work as expected, and it would be deeply inefficient. - store as blobs: maybe. Research required to (a) ensure sqlite compares blobs with correct endianness and (b) has committed to continue to do so. Both seem likely, but conversion to/from might be painful. - store as two values (eg. seconds and nanoseconds), retrieve and reassemble as a pythin INT. On Tuesday, November 24, 2020 at 1:31:21 PM UTC+11 Grunthos wrote: > This of course potentially impacts S3QL **if** it stores FS dates as > numeric values...since SQLite suffers the same limitation. > > On Tuesday, November 24, 2020 at 1:27:42 PM UTC+11 Grunthos wrote: > >> I've added a bug to pyfuse3; the ability for me to consider a patch >> depends in very large part on what level of API compatibility you want to >> break! >> >> Since 64 bits is insufficient to represent a nano-second timestamp, >> something has to become incompatible. I think the choice is down to: >> >> - add numpy (or similar) to the pyfuse dependencies >> - remove the *_ns properties (or at least make them return a >> struct/object instead) >> >> On Monday, November 23, 2020 at 10:42:47 PM UTC+11 Grunthos wrote: >> >>> Further, if my analysis is correct, then it suggests anything that uses >>> the pyfuse composite (double/float) field may have breakage. >>> >>> On Monday, November 23, 2020 at 10:30:52 PM UTC+11 Grunthos wrote: >>> >>>> *definitely* looks like a rounding problem with using a >>>> double-precision value to represent seconds + ns since 1970. For a >>>> standard >>>> double, only 6 decimal digits work...for ns, you need 9 places. >>>> >>>> Try this in python: >>>> >>>> >>> 1577880000.9999999 >>>> >>>> <- this is the source of the problem. >>>> >>> -- You received this message because you are subscribed to the Google Groups "s3ql" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/s3ql/77d7a7cc-82c8-4799-9e8a-125389a6e4f4n%40googlegroups.com.
