devinjdangelo commented on PR #8604: URL: https://github.com/apache/arrow-datafusion/pull/8604#issuecomment-1867009977
> Is there any way to specify the number of output files? Yes, datafusion.execution.minimum_parallel_output_files and datafusion.execution.soft_max_rows_per_output_file control the number of files output (when not writing hive partitions, otherwise 1 file is written per partition). These are session level configs and are currently not configurable per table or per COPY statement, but they could be added as statement level options. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
