devinjdangelo commented on code in PR #7283:
URL: https://github.com/apache/arrow-datafusion/pull/7283#discussion_r1295125065
##########
datafusion/expr/src/logical_plan/builder.rs:
##########
@@ -232,6 +233,23 @@ impl LogicalPlanBuilder {
Self::scan_with_filters(table_name, table_source, projection, vec![])
}
+ /// Create a [CopyTo] for copying the contents of this builder to the
specified file(s)
+ pub fn copy_to(
+ input: LogicalPlan,
+ output_url: String,
+ file_format: OutputFileFormat,
+ per_thread_output: bool,
Review Comment:
There is documentation for these parameters here:
https://github.com/devinjdangelo/arrow-datafusion/blob/27e062a880d8960a9aef4657bc7416d98c9a744c/datafusion/expr/src/logical_plan/dml.rs#L30-L43
I envision most additional options being passed in optionally via the
options Vec<>. I think the [list supported by
DuckDB](https://duckdb.org/docs/sql/statements/copy.html) would be a good
starting point.
It might make sense to implement a CopyOptions struct as you show with a
CopyOptions::from(Vec<String,String>) method. Then, for the DataFrame::write_*
API the user could construct CopyOptions directly rather than passing a
Vec<String,String>, which is an awkward interface in comparison for use
directly from rust code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
