westonpace commented on a change in pull request #12267:
URL: https://github.com/apache/arrow/pull/12267#discussion_r792945719
##########
File path: cpp/src/arrow/compute/exec/options.h
##########
@@ -52,6 +52,19 @@ class ARROW_EXPORT SourceNodeOptions : public
ExecNodeOptions {
std::function<Future<util::optional<ExecBatch>>()> generator;
};
+/// \brief Adapt an Table as a source node
+///
+/// plan->exec_context()->executor() will be used to parallelize pushing to
+/// outputs, if provided.
+class ARROW_EXPORT TableSourceNodeOptions : public ExecNodeOptions {
+ public:
+ TableSourceNodeOptions(std::shared_ptr<Table> table, int64_t max_chunksize)
+ : table(table), max_chunksize(max_chunksize) {}
+
+ std::shared_ptr<Table> table;
+ int64_t max_chunksize;
Review comment:
Can we use `batch_size` instead of `max_chunksize`? That will help make
it clear that this is an analogue of `ScanNodeOptions::ScanNode::batch_size`.
Also, in my mental model, I think of a "chunk" as a bunch of bytes (or a
subsection of an array) and a "batch" as a bunch of rows (I think we're fairly
consistent with this in Arrow but not sure 100%)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
