lirui-apache commented on a change in pull request #12573:
URL: https://github.com/apache/flink/pull/12573#discussion_r438100255
##########
File path:
flink-connectors/flink-connector-hive/src/main/java/org/apache/flink/connectors/hive/read/DirectoryMonitorDiscovery.java
##########
@@ -41,18 +44,36 @@
Context context, long previousTimestamp) throws
Exception {
FileStatus[] statuses = getFileStatusRecurse(
context.tableLocation(),
context.partitionKeys().size(), context.fileSystem());
+ List<Tuple2<List<String>, Long>> partValueList =
suitablePartitions(context, previousTimestamp, statuses);
+
List<Tuple2<Partition, Long>> partitions = new ArrayList<>();
+ for (Tuple2<List<String>, Long> tuple2 : partValueList) {
+ context.getPartition(tuple2.f0).ifPresent(
+ partition -> partitions.add(new
Tuple2<>(partition, tuple2.f1)));
+ }
+ return partitions;
+ }
+
+ @VisibleForTesting
+ static List<Tuple2<List<String>, Long>> suitablePartitions(
Review comment:
Add some comments for this method? What is a suitable partition?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
