danny0405 commented on code in PR #19242:
URL: https://github.com/apache/hudi/pull/19242#discussion_r3575338385
##########
hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/io/storage/row/HoodieFlinkLanceArrowUtils.java:
##########
@@ -113,63 +123,77 @@ public static void writeValue(LogicalType type,
FieldVector vector, int rowId, R
}
public static void writeValue(LogicalType type, FieldVector vector, int
rowId, RowData rowData, int ordinal, boolean utcTimestamp) {
- if (rowData.isNullAt(ordinal)) {
+ Object value = RowData.createFieldGetter(type,
ordinal).getFieldOrNull(rowData);
Review Comment:
Yes. `RowDataArrowWriter.write` invokes this once per field per record.
`createFieldGetter` constructs capturing lambdas (nullable types add another
wrapper), and the `Object` return also boxes primitive values, so this
regresses all existing primitive Lance writes rather than only nested writes.
Could we prebuild/cache a recursive writer/accessor tree once from `rowType`
(including the nested `FieldGetter`s and `ElementGetter`s), or otherwise keep
the specialized primitive accessors in the hot loop?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
