zeroshade commented on issue #1847: URL: https://github.com/apache/arrow-adbc/issues/1847#issuecomment-2115514377
@joellubi I did a similar investigation and proving it. It appears to be caused by the existence of a row group with 0 rows in the file. We can work around it by using `WriteBuffered` instead of `Write` in bulk_ingestion.go. That supports my hypothesis that it is caused by the existence of an empty row group in the file. We should make the change on our end to use `WriteBuffered` which shouldn't cause any significant difference in memory usage due to the file size limitations we use. But we should also bring this issue up with snowflake in the more general case. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
