cisaacson commented on issue #6412: URL: https://github.com/apache/arrow-rs/issues/6412#issuecomment-2377905787
@alamb I learned more about this. I did create a test in `arrow-flight` that returns an empty `RecordBatch`, sure enough it definitely returns it to the client. In looking more I found that the DataFusion `cli` has an empty batch test, if it has an empty set of `RecordBatch`es, it uses the schema to create and return an empty `RecordBatch`. This indicates that DataFusion itself is doing a similar filter. There are many cases with this type of code: `.filter(|b| b.num_rows() > 0)` testing each `RecordBatch`. I'm not sure which one is filtering it out in my case but that must be what is happening. The conclusion is that `arrow-flight-sql` is doing exactly the right thing, the issue is in fact in DataFusion. Based on this I have a relatively easy work-around, similar to how the DataFusion `cli` handles it. I'll close this case now, but if anyone wants to research why DataFusion filters out empty batches, and if that is necessary, it would be good to know. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
