SmirAlex commented on code in PR #20919:
URL: https://github.com/apache/flink/pull/20919#discussion_r1033131501
##########
flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/runtime/functions/table/lookup/fullcache/inputformat/InputFormatCacheLoader.java:
##########
@@ -107,7 +109,11 @@ protected void reloadCache() throws Exception {
} catch (InterruptedException ignored) { // we use interrupt to close
reload thread
} finally {
if (cacheLoadTaskService != null) {
+ // if main cache reload thread encountered an exception,
+ // it interrupts underlying InputSplitCacheLoadTasks threads
cacheLoadTaskService.shutdownNow();
Review Comment:
To be honest, during development I didn't think about relying on
ForkJoinPool. As I understand, it's very useful when we have many little tasks,
that also can be created in already running tasks. But in our case we have
fixed amount ( = number of splits) of long running tasks, that won't create
other subtasks. So simple fixed thread pool looked as a pretty straightforward
way to implement cache loading. Plus, if we will use commonForkPool, there can
be a situation when we will utilize all threads in this pool (our tasks are
long-running), and other tasks that relies on commonForkPool with be starving,
which is undesirable behavior, as I understand. Correct me, if I'm wrong, please
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
