pvary commented on code in PR #7661:
URL: https://github.com/apache/iceberg/pull/7661#discussion_r1222888642
##########
flink/v1.17/flink/src/test/java/org/apache/iceberg/flink/source/SplitHelpers.java:
##########
@@ -62,8 +62,15 @@ public static List<IcebergSourceSplit>
createSplitsFromTransientHadoopTable(
final String warehouse = "file:" + warehouseFile;
Configuration hadoopConf = new Configuration();
final HadoopCatalog catalog = new HadoopCatalog(hadoopConf, warehouse);
+ ImmutableMap<String, String> properties =
ImmutableMap.of(TableProperties.FORMAT_VERSION, "2");
Review Comment:
Since we are moving forward to make V2 the default, we might want to change
the default to `2`, but kept it as `1` for now, since the goal of this PR is
different
##########
flink/v1.17/flink/src/test/java/org/apache/iceberg/flink/source/reader/TestIcebergSourceReader.java:
##########
@@ -55,13 +56,67 @@ public void testReaderMetrics() throws Exception {
TestingReaderOutput<RowData> readerOutput = new TestingReaderOutput<>();
TestingMetricGroup metricGroup = new TestingMetricGroup();
TestingReaderContext readerContext = new TestingReaderContext(new
Configuration(), metricGroup);
- IcebergSourceReader reader = createReader(metricGroup, readerContext);
+ IcebergSourceReader reader = createReader(metricGroup, readerContext,
null);
reader.start();
testOneSplitFetcher(reader, readerOutput, metricGroup, 1);
testOneSplitFetcher(reader, readerOutput, metricGroup, 2);
}
+ @Test
+ public void testReaderOrder() throws Exception {
+ // Create 2 splits
+ List<List<Record>> recordBatchList1 =
+ ReaderUtil.createRecordBatchList(0L, TestFixtures.SCHEMA, 1, 1);
+ CombinedScanTask task1 =
+ ReaderUtil.createCombinedScanTask(
+ recordBatchList1, TEMPORARY_FOLDER, FileFormat.PARQUET,
appenderFactory);
+
+ List<List<Record>> recordBatchList2 =
+ ReaderUtil.createRecordBatchList(1L, TestFixtures.SCHEMA, 1, 1);
+ CombinedScanTask task2 =
+ ReaderUtil.createCombinedScanTask(
+ recordBatchList2, TEMPORARY_FOLDER, FileFormat.PARQUET,
appenderFactory);
+
+ // Sort the splits in one way
+ List<RowData> rowData1 =
Review Comment:
renamed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
