JingsongLi commented on code in PR #1267:
URL: https://github.com/apache/paimon-rust/pull/1267#discussion_r4478989345


##########
crates/paimon/src/io/cache/mod.rs:
##########
@@ -188,10 +193,28 @@ impl LocalCache {
         BlockKey::with_namespace(&self.namespace, path, self.block_size, 
block_index)
     }
 
-    pub(super) fn is_cacheable(&self, path: &str) -> bool {
+    /// Extension exclusions must not disable invalidation of existing shared 
blocks.
+    pub(super) fn is_whitelisted(&self, path: &str) -> bool {
         !FileType::is_mutable(path) && 
self.whitelist.contains(&FileType::classify(path))
     }
 
+    pub(super) fn is_cacheable(&self, path: &str) -> bool {
+        if !self.is_whitelisted(path) {
+            return false;
+        }
+        if self.excluded_extensions.is_empty() {
+            return true;
+        }
+        let name = path.rsplit('/').next().unwrap_or(path);

Review Comment:
   [P2] Match excluded extensions on the storage path actually opened
   
   This checks the raw caller path. The built-in filesystem backend normalizes 
trailing whitespace in OpenDAL, so `payload.blob ` opens the same object as 
`payload.blob` but is classified with an extension of `blob `. With 
`local-cache.whitelist=data` and `local-cache.exclude-extensions=blob`, the 
alias still fills and hits the block cache.
   
   I reproduced it through the public FileSystemCatalog/FileIO path on the 
local merge with main. Both spellings read the same initial bytes. After 
changing the source directly to distinguish cache hits, the canonical path read 
the replacement bytes and the alias returned the cached original. A path 
supplied by a FileIO caller can therefore cache the BLOB body the catalog 
excluded.
   
   Please apply extension matching after the same path normalization used for 
the storage read, or reject aliases that resolve to a different file name. 
Cover the alias and canonical path in the exclusion integration tests.
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to