BlakeOrth commented on issue #19055: URL: https://github.com/apache/datafusion/issues/19055#issuecomment-3608361345
@alamb Can you help me understand the benefit of flattening the `Vec<ObjectMeta>` into individual rows? I'm concerned this could make the output table difficult to read (the `s3://clickhouse-public-datasets/hits_compatible/athena_partitioned/` dataset would result in 100 rows, for instance) and I'm not sure what benefit it would provide since the TTL and metadata size for each element of the `Vec` will be, by and large, uniform. Additionally the cache (at least currently) evicts the entire `Vec` as opposed to individual entries, so we wouldn't expect any fragmentation issues with entries within a collection of metadatas. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
