voonhous commented on code in PR #19495:
URL: https://github.com/apache/hudi/pull/19495#discussion_r3709388212
##########
hudi-trino/src/main/java/io/trino/plugin/hudi/util/PrefilledColumnValues.java:
##########
@@ -108,6 +113,24 @@ public Block toRleBlock(HiveColumnHandle columnHandle, int
positionCount)
}
private Object nativeValueOf(HiveColumnHandle columnHandle)
+ {
+ // Every input to computeNativeValue() is a constant of the split, but
appendTo is called once per
+ // prefilled column per record, and computing re-parses the partition
string each time
+ // ($file_modified_time even formats a timestamp and parses it
straight back). Memoize per column so
+ // each one is resolved once per split. Keyed on the name rather than
the handle because
+ // HiveColumnHandle.hashCode hashes seven fields through a varargs
array, whereas a String caches
+ // its hash. containsKey rather than a null check: null is a
legitimate resolved value, both for
+ // the hive-null convention and for the lenient fallback below.
+ String name = columnHandle.getName();
+ if (resolvedValues.containsKey(name)) {
Review Comment:
Applied rather than deferred, it's contained:
```java
Object value = resolvedValues.getOrDefault(name, UNRESOLVED);
if (value == UNRESOLVED) {
value = computeNativeValue(columnHandle);
resolvedValues.put(name, value);
}
return value;
```
One lookup on the hit path, the one taken per record. The map still stores
real nulls, so it stays a `HashMap` rather than a `ConcurrentHashMap`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
