hililiwei commented on a change in pull request #3817:
URL: https://github.com/apache/iceberg/pull/3817#discussion_r776770808
##########
File path:
flink/v1.14/flink/src/main/java/org/apache/iceberg/flink/source/FlinkSplitGenerator.java
##########
@@ -38,7 +48,22 @@ private FlinkSplitGenerator() {
List<CombinedScanTask> tasks = tasks(table, context);
FlinkInputSplit[] splits = new FlinkInputSplit[tasks.size()];
for (int i = 0; i < tasks.size(); i++) {
- splits[i] = new FlinkInputSplit(i, tasks.get(i));
+ Set<String> hosts = Sets.newHashSet();
+ CombinedScanTask combinedScanTask = tasks.get(i);
+ combinedScanTask.files().forEach(fileScanTask -> {
+ try {
+ final FileSystem fs = new
HadoopFileSystem(DistributedFileSystem.get(new Configuration()));
Review comment:
- I added this flag to the Build of FlinkSource and determined the
value of this flag based on the FS and configuration. But I'm quit not sure
it's appropriate to add a flag there.
- `Util.blockLocations` is indeed a better choice.
PTAL, thx.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
