Le 24/09/2026 à 20:11, Russell Spitzer a écrit :
I think something I haven't seen yet is the clear use-case for non-finite values within a vector type. There has been a lot of theoretical discussion here about use cases but what toolings currently produce these vectors and how do they function in downstream engines? I'd like to limit introducing types that have a wide range of values which are simply are failure modes of different flavors
"Vector" has a more general sense than ML and it's quite trivial to produce vectors with non-finite values using e.g. NumPy or Pandas (or any other piece of dataframe or scientific computing software).
It's not like the general concept of a vector is a IETF standard or anything, so analogies with JSON or UTF8 (where the values are expected to be valid according to the respective specs) do not apply.
This discussion would IMHO have a better chance of converging if we separated the two orthogonal concerns of:
1. representing fixed-length list/vector data in Parquet (with a potential limitation to non-nested types)
2. representing the information that a given column (or column chunk, etc.) does not contain non-finite values
... both of which are useful on their own right, but do not have to be conflated and bundled together.
Regards Antoine.
