korowa commented on issue #13821: URL: https://github.com/apache/datafusion/issues/13821#issuecomment-2594590175
@kosiew the root cause of the issue is how arrow writer handles data for `Dictionary(Decimal)`, and I suppose it'll mostly be fixed by https://github.com/apache/arrow-rs/pull/6987 Regarding RG pruning not being enabled in case literal and column have different datatypes -- I think adding some safe casting to enable it would be a useful feature. For this, perhaps, [PruningPredicate](https://github.com/apache/datafusion/blob/main/datafusion/physical-optimizer/src/pruning.rs#L529) and its interactions with ParquetExec will be a good place to start dig in. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
