Lucas Pickup created ARROW-1436: ----------------------------------- Summary: PyArrow Timestamps written to Parquet as INT96 appear in Spark as 'bigint' Key: ARROW-1436 URL: https://issues.apache.org/jira/browse/ARROW-1436 Project: Apache Arrow Issue Type: Bug Components: Format, Python Affects Versions: 0.6.0 Reporter: Lucas Pickup
When using the 'use_deprecated_int96_timestamps' option to write Parquet files compatible with Spark <2.2.0 (which doesn't support INT64 backed Timestamps) Spark identifies the Timestamp columns as BigInts. Some metadata may be missing. -- This message was sent by Atlassian JIRA (v6.4.14#64029)