alamb opened a new issue #1651: URL: https://github.com/apache/arrow-datafusion/issues/1651
@thinkharderdev added the ability to read from multiple parquet files that have different schemas in https://github.com/apache/arrow-datafusion/pull/1622 However, if some of the files have "incompatible" schemas (e.g. a different type), the data can be ignored from that file. Expected behavior would be to have the plan actually return an error rather than silently ignored. > the issue is that the panic is on the reader thread. In effect it basically just causes the read on that partition to get dropped. We could test for a panic by calling `read_partition` directly from the test if you'd rather do it that way. _Originally posted by @thinkharderdev in https://github.com/apache/arrow-datafusion/pull/1622#discussion_r790151401_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
