robert3005 commented on code in PR #10723:
URL: https://github.com/apache/paimon/pull/10723#discussion_r4492118788


##########
paimon-python/pypaimon/filesystem/vortex_utils.py:
##########
@@ -67,3 +69,97 @@ def to_vortex_specified(file_io: FileIO, file_path: str) -> 
Tuple[str, Optional[
         file_path_for_vortex = file_path_for_vortex.replace('oss://', 's3://')
 
     return file_path_for_vortex, store_kwargs
+
+
+# Paimon never rewrites a data file in place, and every data file has a unique 
name. So a footer,
+# read once, stays valid for the path, and a later open of the same file needs 
no footer IO.
+_FOOTER_CACHE_SIZE = 1024
+_footer_cache: "OrderedDict[str, Any]" = OrderedDict()
+_store_cache: Dict[Tuple[str, Tuple[Tuple[str, str], ...]], Any] = {}
+_cache_lock = threading.Lock()

Review Comment:
   I have pushed a fix for it that registers locks on forking



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to