abstractdog opened a new issue, #27579:
URL: https://github.com/apache/datafusion/issues/27579

   `FileMetadataCache` is keyed by the file's `object_store::path::Path`, which 
is only unique within one object store, while one cache is shared by every 
store of a `RuntimeEnv`. Entries are validated only by size and last-modified 
time, so files with the same in-store path, size and mtime in different stores 
(e.g. `s3://bucket-a/t/x.parquet` and `s3://bucket-b/t/x.parquet`) are served 
each other's Parquet metadata (schema, statistics, row-group offsets), leading 
to wrong pruning or decode errors.
   
   This also blocks sharing one file metadata cache across runtimes, e.g. an 
executor-lifetime cache in Ballista executors that serve many sessions reading 
different buckets.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to