ozankabak commented on code in PR #6049:
URL: https://github.com/apache/arrow-datafusion/pull/6049#discussion_r1179692890
##########
datafusion/core/src/physical_plan/memory.rs:
##########
@@ -223,15 +245,365 @@ impl RecordBatchStream for MemoryStream {
}
}
+/// Execution plan for writing record batches to an in-memory table.
+pub struct MemoryWriteExec {
Review Comment:
This is simply a baby step for implementing things like `COPY TO`s, `INSERT
INTO`s and functionality like that. In practical cases, these things will write
their outputs through a `TableProvider` like a `ListingTable`. How do we test
what that will look like and iterate on such functionality? Basically, this is
the simplest writer that one can perform end-to-end (read-transform-write)
experiments with, its main intention is for anyone to use as a basis/example
for implementing their write operators. I hope it makes a little more sense now.
> If you guys feel strongly that this is the right and maintainable solution
going forward I will defer to your judgement, but from where I see it just adds
complexity / technical debt for no reason.
Thank you -- rest assured we will be maintaining and helping with related
work going forward.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
