thisisnic opened a new issue, #2715: URL: https://github.com/apache/orc/issues/2715
I was closing out some old Arrow issues and this one (apache/arrow#21471, from 2019) looks like it is potentially an ORC bug rather than an Arrow one. I don't understand this domain well myself, so everything below the line was produced by Claude working through the ORC and Arrow source with me. I have checked that the reprex fails against current `main` (4dbfe288) as described, but I have not independently verified the code reading, so please treat it accordingly. I have tried to keep it concise. --- When the JVM's default zone is a fixed offset, the Java writer records a custom ID such as `GMT-00:00` or `GMT+05:30` as the stripe `writerTimezone` (`WriterImpl.java:563` uses `TimeZone.getDefault().getID()`). The Java reader accepts these because `TimeZone.getTimeZone` parses `GMT±hh:mm` (`TreeReaderFactory.java:1251`). The C++ reader resolves a writer timezone only by opening `$TZDIR/<name>` (`getTimezoneByName` in `c++/src/Timezone.cc`), plus the `US/*` alias map from ORC-2011. No such file exists for `GMT-00:00`, so any stripe with a timestamp column fails: ``` $ orc-contents gmt-0000.orc Caught exception in gmt-0000.orc: Time zone file /usr/share/zoneinfo/GMT-00:00 does not exist. Please install IANA time zone database and set TZDIR env. ``` Adding a `GMT-00:00` symlink to `GMT` under `TZDIR` makes the same file read fine, so the timezone name is the only problem with it. Reprex: `gmt-0000.orc` (base64 below, 250 bytes) has one stripe, one timestamp column, uncompressed, with `writerTimezone` set to `GMT-00:00`. It was written with the C++ writer via pyorc (with `TZDIR` pointed at a directory containing a `GMT-00:00` zone file) to reproduce the footer field that Hive produced in the original report. ``` T1JDCgYSBAgBUAAKHgoEAAAAABIWCAFKEBjA2u/vslogwNrv77JaMAFQAHYAD9kwyEAAAAoGCAYQABgICgYIBhABGCAKBggBEAEYBgoGCAUQARgDEgQIABAAEgQIAhAAGglHTVQtMDA6MDAKHgoECAFQAAoWCAFKEBjA2u/vslogwNrv77JaMAFQAAgDEGgaCggDECgYCSA3KAEiDwgMEgEBGgJ0cyAAKAAwACIICAkgACgAMAAwAToECAFQADoWCAFKEBjA2u/vslogwNrv77JaMAFQAECQTkgBYgUyLjEuNAhXEAAYgIAEIgIADCggMAaC9AMDT1JDFw== ``` Suggested fix (untested): recognise names matching `GMT[+-]hh(:mm)?` in `getTimezoneByFilename` and return a fixed-offset `Timezone` instead of loading a file. The `Etc/GMT±n` zoneinfo files are not a substitute since they only cover whole hours and use the inverted POSIX sign. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
