lidavidm commented on code in PR #14151:
URL: https://github.com/apache/arrow/pull/14151#discussion_r993371689
##########
java/dataset/src/main/java/org/apache/arrow/dataset/file/JniWrapper.java:
##########
@@ -45,4 +46,21 @@ private JniWrapper() {
*/
public native long makeFileSystemDatasetFactory(String uri, int fileFormat);
+ /**
+ * Write all record batches in a {@link NativeRecordBatchIterator} into
files. This internally
+ * depends on C++ write API: FileSystemDataset::Write.
+ *
+ * @param itr iterator to be used for writing
+ * @param schema serialized schema of output files
+ * @param fileFormat target file format (ID)
+ * @param uri target file uri
+ * @param partitionColumns columns used to partition output files
+ * @param maxPartitions maximum partitions to be included in written files
+ * @param baseNameTemplate file name template used to make partitions. E.g.
"dat_{i}", i is current partition
+ * ID around all written files.
+ */
+ public native void writeFromScannerToFile(CRecordBatchIterator itr, long
schema_address,
Review Comment:
No, what I mean is that you do not need the iterator at all. Implement the
ArrowReader API for Scanner or whatever the Java-side data source is, then
export that _once_ and import it _once_ on the C++ side. The JNI side should
not need to know about any new Java-side classes or methods.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
