the-other-tim-brown commented on code in PR #12982:
URL: https://github.com/apache/hudi/pull/12982#discussion_r2023884117
##########
hudi-common/src/main/java/org/apache/hudi/common/table/view/HoodieTableFileSystemView.java:
##########
@@ -398,14 +398,24 @@ protected Option<Pair<String, CompactionOperation>>
getPendingLogCompactionOpera
@Override
protected boolean isPartitionAvailableInStore(String partitionPath) {
- return partitionToFileGroupsMap.containsKey(partitionPath);
+ try {
+ readLock.lock();
+ return partitionToFileGroupsMap.containsKey(partitionPath);
+ } finally {
+ readLock.unlock();
+ }
}
@Override
protected void storePartitionView(String partitionPath,
List<HoodieFileGroup> fileGroups) {
- LOG.debug("Adding file-groups for partition :" + partitionPath + ",
#FileGroups=" + fileGroups.size());
- List<HoodieFileGroup> newList = new ArrayList<>(fileGroups);
- partitionToFileGroupsMap.put(partitionPath, newList);
+ try {
Review Comment:
`addFilesToView` and `ensurePartitionLoadedCorrectly` will update the store
which is a map or spillable map for the most part. These were previously using
a `read` lock instead of a `write`. This lead to the ability to update the map
during a `read` which lead to the break in thread-safety.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
