swuferhong commented on PR #20994: URL: https://github.com/apache/flink/pull/20994#issuecomment-1273094223
> solution I think filesystem lookup table source just make sense to small files. For this first commit, I realized a simple `PartitionFetcher` which can only fetch all file partitions, but it can fetch one bulk of partitions after improving `PartitionFetcher` and support partition push down. This solution can reduce the read data size. (BTW, I think reading partial data into cache is difficult for file system source, so it need to read all data into memory at once). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
