2010YOUY01 commented on PR #16734: URL: https://github.com/apache/datafusion/pull/16734#issuecomment-3067676557
> hi @2010YOUY01 > > Do you mean to this PR should be narrowed to only split MemTable datasources by extending #15409? > > > ... some data sources may produce much larger batches. For example, [MemTable](https://docs.rs/datafusion/latest/datafusion/catalog/struct.MemTable.html) provides the RecordBatches it was constructed with -- no additional splitting is done > > Per #16717, this could apply to other sources too. I see, I agree it's great to be more general. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
