github-actions[bot] commented on code in PR #65580:
URL: https://github.com/apache/doris/pull/65580#discussion_r3577518259
##########
be/src/format_v2/parquet/parquet_file_context.cpp:
##########
@@ -192,10 +192,12 @@ std::string build_page_cache_file_key(const
io::FileReader& file_reader,
const io::FileDescription&
file_description) {
const int64_t mtime =
file_description.mtime != 0 ? file_description.mtime :
file_reader.mtime();
- if (mtime == 0) {
- // StoragePageCache is process-global. A key with only path + unknown
mtime can outlive a
- // rewritten local test file, or any external file whose version was
not propagated. Disable
- // v2 parquet page cache until the scan descriptor carries a stable
object version.
+ if (mtime == 0 && !file_description.is_immutable) {
Review Comment:
This still needs a storage-identity component before enabling the `mtime=0`
page-cache key for object stores. For the plugin-driven Hudi/Paimon ranges,
`getModificationTime()` defaults to 0, and `S3FileReader::mtime()` is also 0;
when these readers mark the split immutable, this code builds a process-global
key from only `fs_name`, `s3://bucket/key`, `mtime=0`, and size. The actual S3
client, though, is created from the scan properties/endpoint in `FileFactory`,
and FE only populates `fs_name` for HDFS. Two catalogs or S3-compatible
endpoints that both expose `s3://bucket/key` with the same size can therefore
read cached Parquet page bytes from the other endpoint. Please either include a
stable non-secret object-store identity in this key, or keep page cache
disabled for unknown-mtime object-store files until such an identity is
available.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
