rdblue commented on a change in pull request #1237:
URL: https://github.com/apache/iceberg/pull/1237#discussion_r460563855
##########
File path: flink/src/test/java/org/apache/iceberg/flink/data/RandomData.java
##########
@@ -88,20 +102,151 @@ public Row next() {
};
}
+ private static Iterable<RowData> generateRowData(Schema schema, int
numRecords,
+
Supplier<RandomRowDataGenerator> supplier) {
+ return () -> new Iterator<RowData>() {
+ private final RandomRowDataGenerator generator = supplier.get();
+ private int count = 0;
+
+ @Override
+ public boolean hasNext() {
+ return count < numRecords;
+ }
+
+ @Override
+ public RowData next() {
+ if (!hasNext()) {
+ throw new NoSuchElementException();
+ }
+ ++count;
+ return (RowData) TypeUtil.visit(schema, generator);
+ }
+ };
+ }
+
+ public static Iterable<RowData> generateRowData(Schema schema, int
numRecords, long seed) {
+ return generateRowData(schema, numRecords, () -> new
RandomRowDataGenerator(seed));
+ }
+
public static Iterable<Row> generate(Schema schema, int numRecords, long
seed) {
return generateData(schema, numRecords, () -> new
RandomRowGenerator(seed));
}
- public static Iterable<Row> generateFallbackData(Schema schema, int
numRecords, long seed, long numDictRows) {
- return generateData(schema, numRecords, () -> new FallbackGenerator(seed,
numDictRows));
+ public static Iterable<RowData> generateFallbackData(Schema schema, int
numRecords, long seed, long numDictRows) {
+ return generateRowData(schema, numRecords, () -> new
FallbackGenerator(seed, numDictRows));
}
- public static Iterable<Row> generateDictionaryEncodableData(Schema schema,
int numRecords, long seed) {
- return generateData(schema, numRecords, () -> new
DictionaryEncodedGenerator(seed));
+ public static Iterable<RowData> generateDictionaryEncodableData(Schema
schema, int numRecords, long seed) {
+ return generateRowData(schema, numRecords, () -> new
DictionaryEncodedGenerator(seed));
}
- private static class RandomRowGenerator extends
RandomGenericData.RandomDataGenerator<Row> {
+ private static class RandomRowDataGenerator extends
TypeUtil.CustomOrderSchemaVisitor<Object> {
Review comment:
I think this should use the generator for generics and validate against
generic data. That will catch more cases.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
