JingsongLi commented on code in PR #11245:
URL: https://github.com/apache/paimon/pull/11245#discussion_r4623836178
##########
paimon-api/src/main/java/org/apache/paimon/CoreOptions.java:
##########
@@ -5135,19 +5135,13 @@ public boolean withVectorFormat() {
}
public Set<String> vectorField() {
- String vectorFields = options.get(CoreOptions.VECTOR_FIELD);
- if (vectorFields == null || vectorFields.trim().isEmpty()) {
- return Collections.emptySet();
- }
- return
Arrays.stream(vectorFields.trim().split(",")).collect(Collectors.toSet());
+ return parseCommaSeparatedSet(VECTOR_FIELD);
Review Comment:
[P2] Preserve exact column names when upgrading existing tables
`vector-field='v1, v2'` was valid with vector columns named `v1` and ` v2`
(the latter genuinely starts with a space). I created and wrote a real Data
Evolution Parquet table with that schema using the baseline parser. The
baseline planned one split. This head still loads the stored field names but
fails `newScan().plan()` with `Some of the columns specified as vector-field
are unknown` because runtime validation now looks for `v2`. Please resolve an
exact token against the actual schema before trying its trimmed form, and use
the same resolution for validation and connector type conversion. Include an
existing-table upgrade case alongside the new whitespace tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]