danny0405 commented on code in PR #18965:
URL: https://github.com/apache/hudi/pull/18965#discussion_r3392737166
##########
hudi-common/src/main/java/org/apache/hudi/metadata/HoodieTableMetadataUtil.java:
##########
@@ -2499,8 +2501,39 @@ public static HoodieRecordGlobalLocation
getLocationFromRecordIndexInfo(
fileId = originalFileId;
}
- final java.util.Date instantDate = new java.util.Date(instantTime);
- return new HoodieRecordGlobalLocation(partition,
HoodieInstantTimeGenerator.formatDate(instantDate), fileId);
+ return new HoodieRecordGlobalLocation(partition,
formatRecordIndexInstant(instantTime), fileId);
+ }
+
+ // The formatted instant of a record-index entry is a pure function of the
entry's epoch millis
+ // and the JVM default time zone used by
HoodieInstantTimeGenerator.formatDate, so cached strings
+ // are only valid for the zone they were formatted under. The cache is per
thread because
+ // lookups run on concurrent executor task threads and decode per-record
last-write instants,
+ // where a shared slot would just be eviction churn.
+ private static final int FORMATTED_INSTANT_CACHE_MAX_SIZE = 1024;
+ private static final ThreadLocal<FormattedInstantCache>
FORMATTED_INSTANT_CACHE =
+ ThreadLocal.withInitial(FormattedInstantCache::new);
+
+ private static final class FormattedInstantCache {
+ private ZoneId zoneId;
+ private final Map<Long, String> formattedByMillis = new HashMap<>();
+ }
+
+ private static String formatRecordIndexInstant(long instantTimeMillis) {
+ FormattedInstantCache cache = FORMATTED_INSTANT_CACHE.get();
+ ZoneId currentZoneId = ZoneId.systemDefault();
Review Comment:
is the default timezone we always uses to calculate the instantTimeMillis?
BTW, not a fan of cache like this, it is very error prone.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
