Omega359 commented on issue #18411: URL: https://github.com/apache/datafusion/issues/18411#issuecomment-3497725100
> The only problem I see is that it's hard to know up front which of the two you options (view or not) to choose based on the parquet metadata. On a per row group basis you can decide this since the dictionary is entirely available in one page so you can figure out the maximum string length for the row group. But if that choice needs to be made for the entire file it's not so simple. Here too `ENUM` could be a possible heuristic here, but I'm not sure how often that actually gets used in practice. Wouldn't this be even more the case if using ListingTable with many files> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
