voonhous commented on code in PR #19495:
URL: https://github.com/apache/hudi/pull/19495#discussion_r3706453373
##########
hudi-trino/src/main/java/io/trino/plugin/hudi/util/PrefilledColumnValues.java:
##########
@@ -108,6 +113,24 @@ public Block toRleBlock(HiveColumnHandle columnHandle, int
positionCount)
}
private Object nativeValueOf(HiveColumnHandle columnHandle)
+ {
+ // Every input to resolve() is a constant of the split, but appendTo
is called once per prefilled
+ // column per record, and resolving re-parses the partition string
each time ($file_modified_time
+ // even formats a timestamp and parses it straight back). Memoize per
column so each one is
+ // resolved once per split. Keyed on the name rather than the handle
because
+ // HiveColumnHandle.hashCode hashes seven fields through a varargs
array, whereas a String caches
+ // its hash. containsKey rather than a null check: null is a
legitimate resolved value, both for
+ // the hive-null convention and for the lenient fallback below.
+ String name = columnHandle.getName();
+ if (resolvedValues.containsKey(name)) {
+ return resolvedValues.get(name);
+ }
+ Object value = resolve(columnHandle);
+ resolvedValues.put(name, value);
+ return value;
Review Comment:
Renamed. Small correction: `nativeValueOf` is private too -- it's the
memoizing entry point, and this is the uncached computation behind it -- but
the pairing reads better with `computeNativeValue`, so applied.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
