alamb opened a new pull request, #6347: URL: https://github.com/apache/arrow-datafusion/pull/6347
# Which issue does this PR close? Closes https://github.com/apache/arrow-datafusion/issues/6339 # Rationale for this change I want to make it easier for DataFusion to be extended with new write sources. I want to use this both for `COPY ... TO ...` statements as well as @JanKaul is looking into using it for Delta.rs: https://github.com/apache/arrow-datafusion/issues/6339#issuecomment-1545777125 # What changes are included in this PR? Changes: 1. Add DataSink API (thanks @tustvold and @JanKaul for this discussion) 2. Port `MemTable` write to be in terms of DataSink (it was really helpful to have a specific implemetation to try this with -- thanks @metesynnada ) 3. Remove `MemoryWriteExec`, the `MemTable` specific execution plan 3. Update tests (TODO) 4. Ensure there is a test for inserting the wrong schema into a table # Are these changes tested? Yes # Are there any user-facing changes? The API for insert changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
