bvaradar commented on code in PR #9778:
URL: https://github.com/apache/hudi/pull/9778#discussion_r1336623923
##########
hudi-common/src/main/java/org/apache/hudi/common/util/collection/ExternalSpillableMap.java:
##########
@@ -213,6 +213,8 @@ public R put(T key, R value) {
if (this.inMemoryMap.containsKey(key)) {
this.inMemoryMap.put(key, value);
+ } else if (inDiskContainsKey(key)) {
Review Comment:
Thanks for the fix @llincc .
This is definitely one option of keeping subsequent versions of the same
record (identified by key) in disk if the first version goes to disk.
The other option would be : if the record was earlier in disk but based on
current sizing constraints, we allow the new version of the record in in memory
map but only after removing the key from diskBasedMap.
There is difference in performance between the two option. In the second
option, subsequent get() calls on those records will be faster.
Thoughts ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
