Hi, Thanks for looking into this! Let me try to help by clarifying some of the questions that you've raised:
- I think it's fine to break pyfuse3 backwards compatibility, as long as we are explicit about it and increment the major version number. That is what it's there for. - I would rather not break backwards compatibility in S3QL, since this means that we have to write upgrade code and everyone has to upgrade their filesystem. - I do not think S3QL needs to support nanosecond resolution (the VFS does not expect this from filesystems in general either). However, we should do better than seconds. - Currently, S3QL stores data as an int64 of nanoseconds (in both Python and SQLite). I think this should be fine - in 64 bits we can store 9223372036854775808 ns which is more than 292 years (starting at 1970). - pyfuse3 is not intended to use floating point values at all (see e.g. http://www.rath.org/pyfuse3-docs/data.html#pyfuse3.EntryAttributes). My best guess is that code that uses "1e9" should read "1 000 000 000" and that I (or whoever wrote the code) forgot that 1e9 gives a float rather than an integer. All the best, -Nikolaus On Nov 23 2020, Grunthos <[email protected]> wrote: > I've added a bug to pyfuse3; the ability for me to consider a patch depends > in very large part on what level of API compatibility you want to break! > > Since 64 bits is insufficient to represent a nano-second timestamp, > something has to become incompatible. I think the choice is down to: > > - add numpy (or similar) to the pyfuse dependencies > - remove the *_ns properties (or at least make them return a > struct/object instead) > > On Monday, November 23, 2020 at 10:42:47 PM UTC+11 Grunthos wrote: > >> Further, if my analysis is correct, then it suggests anything that uses >> the pyfuse composite (double/float) field may have breakage. >> >> On Monday, November 23, 2020 at 10:30:52 PM UTC+11 Grunthos wrote: >> >>> *definitely* looks like a rounding problem with using a double-precision >>> value to represent seconds + ns since 1970. For a standard double, only 6 >>> decimal digits work...for ns, you need 9 places. >>> >>> Try this in python: >>> >>> >>> 1577880000.9999999 >>> >>> <- this is the source of the problem. >>> >> > > -- > You received this message because you are subscribed to the Google Groups > "s3ql" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/s3ql/0a4a5cb4-bf43-4f72-bacd-69636d26d37en%40googlegroups.com. -- GPG Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F »Time flies like an arrow, fruit flies like a Banana.« -- You received this message because you are subscribed to the Google Groups "s3ql" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/s3ql/87h7pfds31.fsf%40vostro.rath.org.
