JingsongLi commented on code in PR #8334:
URL: https://github.com/apache/paimon/pull/8334#discussion_r3471518255
##########
paimon-core/src/main/java/org/apache/paimon/operation/BaseAppendFileStoreWrite.java:
##########
@@ -163,9 +163,10 @@ public void withWriteType(RowType writeType) {
if (blobContext != null) {
blobContext = blobContext.withWriteType(writeType);
}
- int fullCount = rowType.getFieldCount();
List<String> fullNames = rowType.getFieldNames();
- this.writeCols = writeType.getFieldNames();
+ // writeCols carries (possibly nested) dotted paths, e.g. ["f0",
"nest.a"]; a plain
+ // top-level name means the whole column, a dotted path means only
that sub-field is written
+ this.writeCols = writeType.leafPaths(rowType);
Review Comment:
This can persist writeCols such as `nest.sub.x` for a deeper partial struct,
but the read path below only supports composing one nested level and later
throws when the full row is read (`DataEvolutionSplitRead` rejects
partially-written nested sub-fields deeper than one level). That means a caller
using
`BatchTableWrite.withWriteType(table.rowType().projectByPaths(Collections.singletonList("nest.sub.x")))`
can successfully commit a file that makes normal full-table reads fail
afterwards. Please reject unsupported deeper dotted paths before
writing/committing them, or extend the reader to compose them recursively.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
