Re: [I] Coerce parquet int96 timestamps to microsecond precision [arrow-rs]

2024-04-21 Thread via GitHub
ion-elgreco commented on issue #5655: URL: https://github.com/apache/arrow-rs/issues/5655#issuecomment-2067963356 @mapleFU ![image](https://github.com/apache/arrow-rs/assets/15728914/df18e762-0063-4384-bb78-ca10c7d1ba40) https://spark.apache.org/docs/3.5.1/configuration.html#content

Re: [I] Coerce parquet int96 timestamps to microsecond precision [arrow-rs]

2024-04-17 Thread via GitHub
mapleFU commented on issue #5655: URL: https://github.com/apache/arrow-rs/issues/5655#issuecomment-2061882525 Can you provide the link for Spark's behavior? I found Int96 mentioned here but I wonder how it defines it specifically https://spark.apache.org/docs/latest/sql-data-sources-parquet

Re: [I] Coerce parquet int96 timestamps to microsecond precision [arrow-rs]

2024-04-17 Thread via GitHub
liukun4515 commented on issue #5655: URL: https://github.com/apache/arrow-rs/issues/5655#issuecomment-2061005273 I find the definition of the int96 in the deprecated doc https://github.com/xhochy/parquet-format/blob/cb4727767823ae201fd567f67825cc22834c20e9/LogicalTypes.md#int96-timestamps-al

Re: [I] Coerce parquet int96 timestamps to microsecond precision [arrow-rs]

2024-04-17 Thread via GitHub
liukun4515 commented on issue #5655: URL: https://github.com/apache/arrow-rs/issues/5655#issuecomment-2060992303 > @tustvold to my understanding it's actually microsecond precision but it's saved as a logical int96 I think in the parquet, the physical type of `int96` represent the tim