Hi all, I'd like to start a vote to add the FINITE_VECTOR logical type to the Parquet format, as specified in parquet-format PR 624 [1] (at commit 0596fde). If you'd like to discuss this proposal please use the active discussion thread [2].
FINITE_VECTOR is roughly: - annotates the same structure used for LIST, giving unaware readers a fallback - every non-null vector has exactly num_elements elements - elements are required (non-null) and finite (no NaN, null, +Inf or -Inf) Writers must enforce these rules, and readers may rely on them. This proposal does not add a new physical representation and doesn't preclude one later. This came out of active discussions on the list [2][3] and the community syncs. The type was renamed from VECTOR to FINITE_VECTOR to indicate it's specific requirements and to leave room for a more general vector type. Implementations: - parquet-java: [4] - parquet-cpp: [5] This vote will be open for at least 72 hours. [ ] +1: Add FINITE_VECTOR to the Parquet spec [ ] +0: Not going to block, but prefer ... [ ] -1: Do not add FINITE_VECTOR to Parquet because ... Thanks to everyone who took part in the discussions and reviews! Rok [1] https://github.com/apache/parquet-format/pull/624 [2] https://lists.apache.org/thread/kgmgn2vmrz49thbfhv8b66lglsyd81xt [3] https://lists.apache.org/thread/xot5f3ghhtc82n1bf0wdl9zqwlrzqks3 [4] https://github.com/apache/parquet-java/pull/3876 [5] https://github.com/apache/arrow/pull/55348
