SmirAlex commented on code in PR #20919:
URL: https://github.com/apache/flink/pull/20919#discussion_r1033131501
##########
flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/runtime/functions/table/lookup/fullcache/inputformat/InputFormatCacheLoader.java:
##########
@@ -107,7 +109,11 @@ protected void reloadCache() throws Exception {
} catch (InterruptedException ignored) { // we use interrupt to close
reload thread
} finally {
if (cacheLoadTaskService != null) {
+ // if main cache reload thread encountered an exception,
+ // it interrupts underlying InputSplitCacheLoadTasks threads
cacheLoadTaskService.shutdownNow();
Review Comment:
To be honest, during development I didn't think about relying on
ForkJoinPool. As I understand, it's very useful when we have many little tasks,
which can create other little subtasks. But in our case we have fixed amount (
= number of splits) of long running tasks, that won't create other subtasks. So
simple fixed thread pool looked as a pretty straightforward way to implement
cache loading. Plus, if we will use commonForkPool, there can be a situation
when we will utilize all threads in this pool (our tasks are long-running), and
other tasks that relies on commonForkPool with be starving, which is
undesirable behavior, as I understand. Correct me, if I'm wrong, please
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
