the-other-tim-brown commented on code in PR #12982:
URL: https://github.com/apache/hudi/pull/12982#discussion_r1999905988
##########
hudi-common/src/main/java/org/apache/hudi/common/util/collection/ExternalSpillableMap.java:
##########
@@ -92,7 +92,7 @@ public class ExternalSpillableMap<T extends Serializable, R>
implements Map<T, R
public ExternalSpillableMap(long maxInMemorySizeInBytes, String
baseFilePath, SizeEstimator<T> keySizeEstimator,
SizeEstimator<R> valueSizeEstimator, DiskMapType
diskMapType, CustomSerializer<R> valueSerializer,
boolean isCompressionEnabled, String
loggingContext) throws IOException {
- this.inMemoryMap = new HashMap<>();
+ this.inMemoryMap = new ConcurrentHashMap<>();
Review Comment:
The ExternalSpillableMap is used as part of the `SPILLABLE_DISK` version of
the filesystem view and therefore it is used by the timeline server which can
handle multiple concurrent requests. It can also be called by users looking to
query the table in their own applications built on Hudi.
Currently you can read from the map without obtaining a lock. This can cause
a null to be returned incorrectly at runtime if the map is undergoing a resize
operation concurrently.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
