davisusanibar commented on code in PR #35034:
URL: https://github.com/apache/arrow/pull/35034#discussion_r1163378277
##########
java/dataset/src/test/java/org/apache/arrow/dataset/file/TestFileSystemDataset.java:
##########
@@ -101,6 +101,29 @@ public void testBaseParquetRead() throws Exception {
AutoCloseables.close(factory);
}
+ @Test
+ public void testMultipleParquetReadFromUris() throws Exception {
+ ParquetWriteSupport writeSupport1 =
ParquetWriteSupport.writeTempFile(AVRO_SCHEMA_USER, TMP.newFolder(),
+ 1, "a");
+ ParquetWriteSupport writeSupport2 =
ParquetWriteSupport.writeTempFile(AVRO_SCHEMA_USER, TMP.newFolder(),
+ 2, "b");
+ String expectedJsonUnordered = "[[1,\"a\"],[2,\"b\"]]";
+
+ ScanOptions options = new ScanOptions(1);
+ FileSystemDatasetFactory factory = new
FileSystemDatasetFactory(rootAllocator(), NativeMemoryPool.getDefault(),
+ FileFormat.PARQUET, new String[]{writeSupport1.getOutputURI(),
writeSupport2.getOutputURI()});
+ Schema schema = inferResultSchemaFromFactory(factory, options);
+ List<ArrowRecordBatch> datum = collectResultFromFactory(factory, options);
+
+ assertScanBatchesProduced(factory, options);
+ assertEquals(2, datum.size());
+ datum.forEach(batch -> assertEquals(1, batch.getLength()));
+ checkParquetReadResult(schema, expectedJsonUnordered, datum);
+
+ AutoCloseables.close(datum);
+
+ }
Review Comment:
Could be possible to add testing for exception throws: (1) Invalid URI , (2)
ensure that they all share a FileSystem type
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
