Thank you everyone for the discussion today! The meeting notes can be found here: https://docs.google.com/document/d/1zA8PskNDFKXXJpzWQr25CFoX_aIZHBNHbkxk0t3PteE/edit?tab=t.v399sh5bz8he and below is a TLDR version: - Focus on a 1D numeric vector type with fixed length and non-null, finite coordinates; +0 and -0 remain valid. - Keep the initial spec simple and build on Parquet’s emerging vector representation. - Initially support int, long, float, and double; defer narrower/special types such as int8 as additional Iceberg primitive types (while allowing such parquet representation for vector). - Keep higher level properties such as distance functions out of scope, and revisit them with the index design. Sparse vectors are also out of scope for now: alternative options exist and waiting for concrete use cases. - For formats without native vector support, use a list representation with writer enforcement. Will explore an Avro vector logical type with Avro community; ORC support remains open.
I'll update the spec PR in the next few days (also appreciate the quick feedback from everyone!) Thanks, Yan On Wed, Oct 7, 2026 at 5:31 PM Yan Yan <[email protected]> wrote: > Thank you Steven for helping secure the time slot for the vector type > sync! And thanks everyone for your feedback on the design doc. > > In preparation for tomorrow's discussion, I've created a draft PR > https://github.com/apache/iceberg/pull/18416 for the spec changes based > on the progress in the parquet community on the Vector type proposal [1 > <https://github.com/apache/parquet-format/pull/624>] [2 > <https://docs.google.com/document/d/1nf30OqK_UqxA4YTEZQszmOBEG56m9M5mp9rIYC2SUWc/edit?tab=t.0>] > , and will continue to iterate on both the spec PR and the design doc based > on the discussion feedback. > > Thanks, > Yan > > On Tue, Sep 29, 2026 at 10:10 AM Steven Wu <[email protected]> wrote: > >> Hi, >> >> As some of you are aware, there is an ongoing discussion about a new >> *Vector* type in both the Parquet and Iceberg communities. >> >> Iceberg Vector type design doc (from Yan Yan): >> https://docs.google.com/document/d/1zA8PskNDFKXXJpzWQr25CFoX_aIZHBNHbkxk0t3PteE/edit?tab=t.0 >> >> To accelerate the Vector type discussion, we are reusing/extending >> Thursday's biweekly series to cover both File and Vector types (*starting >> on Oct 8*). Since the file type discussion is reaching consensus, more >> time will likely be spent on the Vector type discussion. The corresponding >> dev calendar event title has been updated. >> >> Thanks, >> Steven >> >
