LuciferYang opened a new pull request, #9642: URL: https://github.com/apache/paimon/pull/9642
### Purpose close #9641 `CachingFileIO.newInputStream` keyed the memory cache by path alone, while the disk branch keys by path, length and modification time. The whitelist that decides what gets cached contains `META`, and `consumer-*` and `service-*` classify as `META` while being exactly the files written in place by `overwriteFileUtf8`. `FileType.isMutable` only excludes `EARLIEST` and `LATEST`, so after a consumer reset the cached blocks keep serving the pre-reset content, with the cached file size pinned per path too. The memory branch now builds the same versioned key the disk branch uses, keeping the namespace prefix that `SharedCacheManager` invalidation matches on, and passes the known length instead of `-1`. Blacklisting the two prefixes in `isMutable` would have been cheaper, and I went the other way deliberately: `TagManager.createOrReplaceTag` overwrites `tag-*`, the Iceberg metadata writes go through `overwriteFileUtf8`, and so does `_SUCCESS`, so a prefix list is something to keep maintaining while a version in the key is not. The cost is one `delegate.getFileStatus` per open in memory mode, a HEAD on an object store. Before this, the size was resolved lazily on first read and then cached per path, so the change is from once per path to once per open; cached blocks are unaffected either way. One limit remains, shared with the disk mode: a delegate with second-granularity modification times can still collide if a rewrite lands in the same second at the same length. ### Tests `CachingFileIOTest.testMemoryModeServesFreshContentAfterInPlaceOverwrite` reads `consumer-1` through a memory-mode `CachingFileIO`, replaces the file in place with different content and a later modification time, and reads again. It asserts the content of both reads and the number of times the delegate was opened, so it pins both freshness and that the first version really was served from cache. Against the unfixed code the second read returns the first version's bytes. `mvn -pl paimon-common -Dtest=CachingFileIOTest test` on JDK 8: 29 tests, 0 failures. `spotless:check` and `checkstyle:check` on paimon-common are clean. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
