zhztheplayer commented on a change in pull request #10114:
URL: https://github.com/apache/arrow/pull/10114#discussion_r771088431
##########
File path:
java/dataset/src/test/java/org/apache/arrow/dataset/ParquetWriteSupport.java
##########
@@ -42,13 +43,15 @@
private final Schema avroSchema;
private final List<GenericRecord> writtenRecords = new ArrayList<>();
private final GenericRecordListBuilder recordListBuilder = new
GenericRecordListBuilder();
+ private final Random random = new Random();
Review comment:
> If we do not use a seed, the file name for each run is identical
I am not sure I get you point here correctly, but did you mean "If we do not
use a seed, the file name for each run is **not** identical"?
##########
File path:
java/dataset/src/test/java/org/apache/arrow/dataset/file/TestFileSystemDataset.java
##########
@@ -295,6 +326,39 @@ public void testMemoryAllocationOnAssociatedAllocator()
throws Exception {
Assert.assertEquals(-expected_diff, finalReservation - reservation);
}
+ private void checkParquetReadResult(Schema schema, String expectedJson,
List<ArrowRecordBatch> actual)
+ throws IOException {
+ final ObjectMapper json = new ObjectMapper();
+ final Set<?> expectedSet = json.readValue(expectedJson, Set.class);
+ final Set<List<Object>> actualSet = new HashSet<>();
+ final int fieldCount = schema.getFields().size();
+ try (VectorSchemaRoot vsr = VectorSchemaRoot.create(schema,
rootAllocator())) {
+ VectorLoader loader = new VectorLoader(vsr);
+ for (ArrowRecordBatch batch : actual) {
+ try {
+ loader.load(batch);
+ int batchRowCount = vsr.getRowCount();
+ for (int i = 0; i < batchRowCount; i++) {
+ List<Object> row = new ArrayList<>();
+ for (int j = 0; j < fieldCount; j++) {
+ Object object = vsr.getVector(j).getObject(i);
+ if (Primitives.isWrapperType(object.getClass())) {
+ row.add(object);
+ } else {
+ row.add(object.toString());
+ }
+ }
+ actualSet.add(row);
+ }
+ } finally {
+ batch.close();
Review comment:
removed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
