ion-elgreco commented on issue #9183: URL: https://github.com/apache/arrow-rs/issues/9183#issuecomment-5172016133
Noted, I have been looking at the repo and where everything can fit in. I have some ideas but before I can start I noticed some issues with leap seconds and other parsing gaps. 1. Chrono supports leap seconds by using nanoseconds to denote there is a leap second, jiff doesn't support it all, it parses `23:59:60` into a clipped `23:59:59` 2. `23:59:60` can be parsed by arrow-rs it turns into a time type (s) holding the value 86400 but the spec defines the values as 86400 exclusive, so that does not seem correct. Writing to parquet also writes the value as-is. https://github.com/apache/arrow-rs/blob/b9d3b08a49dbe13351cfe357c12eb03473267176/format/Schema.fbs#L265-L267 Other writers fail though because converting it to string format fails because the time to chrono time conversion doesn't account that we now can have 86400 value which chrono rejects because its leap second conversion only goes through naivedatetime 4. jiff doesn't handle parsing leap seconds so we can never roundtrip -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
