PFischbeck opened a new pull request, #18676: URL: https://github.com/apache/iceberg/pull/18676
This PR adds a VectorType primitive parameterized by an element type (int, long, float, double) and a dimension (>= 1), serialized as `vector(<element-type>, <dimension>)`, e.g. `vector(int, 768)`. It's gated to format version 4, rejected as an identity partition source, and allows only null defaults. Reading and writing to data files is not covered in this PR. Design discussion: https://docs.google.com/document/d/1zA8PskNDFKXXJpzWQr25CFoX_aIZHBNHbkxk0t3PteE/edit?usp=sharing Spec change PR: https://github.com/apache/iceberg/pull/18416 The API design choice to treat vector element types as a separate enum is for future compatibility. As of now, all supported element types are also Iceberg primitive types. But there are discussions that we might extend this list in the future, even to types like `float16` that might only be supported as element type, not Iceberg primitive type. I'm open for suggestions here (such as only allowing the fixed set of primitive types). My main concern then would be that this is a one-way road to not allowing for other non-Iceberg element types in the future without breaking API changes. AI disclaimer: Cursor Grok 4.6 was used to generate most of the code (with several iterations and feedback loops). All parts were human-reviewed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
