I've had the chance to look into this. There is one issue that came up which I don't know how to handle. Previously, int96 seems to have been used for nanosecond precision, but this is somewhat deprecated, as far as I understand it. So, how should we handle nanoseconds and int96 vs int64 in 1) reading from and b) writing to parquet. There seem to be some writer settings, all related to timestamp precision properties. Is there any advise someone of you can give me in that regard?
Thanks, Roman Von: Roman Karlstetter Gesendet: Freitag, 9. November 2018 08:38 An: dev@arrow.apache.org Betreff: AW: Support for TIMESTAMP_NANOS in parquet-cpp I would be willing to implement that. I’ll probably need some advice on my patch though, as I’m fairly new to the parquet code. Roman Von: Wes McKinney Gesendet: Donnerstag, 8. November 2018 23:22 An: dev@arrow.apache.org Betreff: Re: Support for TIMESTAMP_NANOS in parquet-cpp I opened an issue here https://issues.apache.org/jira/browse/ARROW-3729. Patches would be welcome On Sat, Oct 20, 2018 at 12:55 PM Wes McKinney <wesmck...@gmail.com> wrote: > > hi Roman, > > We would welcome adding such a document to the Arrow wiki > https://cwiki.apache.org/confluence/display/ARROW. As to your other > questions, it really depends on whether there is a member of the > Parquet community who will do the work. Patches that implement any > released functionality in the Parquet format specification are > welcome. > > Thanks > Wes > On Thu, Oct 18, 2018 at 10:59 AM Roman Karlstetter > <roman.karlstet...@gmail.com> wrote: > > > > Hi everyone, > > in parquet-format, there is now support for TIMESTAMP_NANOS: > > https://github.com/apache/parquet-format/pull/102 > > For parquet-cpp, this is not yet supported. I have a few questions now: > > • is there an overview of what release of parquet-format is currently fully > > support in parquet-cpp (something like a feature support matrix)? > > • how fast are new features in parquet-format adopted? > > I think having a document describing the current completeness of > > implementation of the spec would be very helpful for users of the > > parquet-cpp library. > > Thanks, > > Roman > > > >
