Thank you Sagar for putting this all together. I went through the proposal and the code, and I have a few questions from my side that I think are important to clarify before we move forward:
1. *Zero-copy:* I think zero-copy is a good direction, especially for the Fluss → Arrow → Lance path. Could we clarify the scope of this claim? Through Flink SQL, the data may still be materialized into Flink's own representation, so it would be good to distinguish where we can actually guarantee zero-copy. 2. *Dimension as part of the type:* I prefer keeping the dimension in the type itself, e.g. VECTOR(1536), rather than as a table property. This makes dimensionality an explicit schema guarantee and avoids downstream systems having to infer it from the data. 3. *Dimension validation:* What happens when a VECTOR(1536) column receives a vector with 1535 or 1537 elements? We should ideally have a clear validation error at the write boundary rather than a failure later during Arrow/Lance conversion. 4. *Null semantics:* Could we clarify how nulls are handled? In particular, do we support a completely NULL vector, and are individual elements within a vector allowed to be NULL? These are two different cases and may have different implications for Arrow and Lance. I noticed the POC has testNullVectorTiering, so it would be good to understand exactly what case is covered. 5. *Compatibility:* Since this introduces a new type ID, how do older clients behave when they encounter a VECTOR column? We should make sure this is a clear and predictable compatibility failure rather than an opaque error. 6. *Type ID:* The proposal currently mentions something like "e.g. 16". I think we should formally reserve the type ID before merging so there is no possibility of conflicts with other changes. 7. *Nesting/matrices:* Since matrices are explicitly out of scope, can we confirm whether ARRAY<VECTOR(1536)> is actually supported in v1? Otherwise, the suggested workaround of wrapping vectors in an array would not be valid. 8. *Other lakehouse integrations:* It would be good to clarify how VECTOR is expected to behave across the different lakehouse paths, particularly where the hot tier has a fixed-size vector but the cold tier may only support a list representation. We should make sure the type semantics remain consistent across tiering and union reads. Also, I think this would be a great topic to bring to the community call. There are several cross-component implications here, and getting more eyes from the community would help us validate the design and catch any compatibility or integration concerns early. Best Regards, Mehul Batra
