adriangb commented on issue #15920: URL: https://github.com/apache/datafusion/issues/15920#issuecomment-2889346539
So I think this was introduced in dccf3778e95037f3ed8740627799e9d658943157. The issue isn't actually #15780 as I originally thought (and that issue may need some revisiting in general) rather that there is _no_ cast. Previously we passed the table schemas into the pruning infrastructure. That eventually hit [this cast](https://github.com/pydantic/datafusion/blob/76a7789ace33ced54c973fa0d5fc9d1866e1bf19/datafusion/physical-optimizer/src/pruning.rs#L972) which converted the array to the table schema's data type. Since we now pass in the file's phyiscal schema for pruning this cast doesn't do anything and thus this error is hit. I think the solution should be to add casts for the literals into the file's data type. This avoids casting the data itself, which is especially important for row level predicate pushdown where that would be very expensive. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
