dcausse added a comment.
Wanting the Executor to free of any previous task is perhaps something we want to reconsider. Ideally when we fetch 2 revisions at the same time if one fetch fails we want to fail (interrupt) the second fetch as soon as possible. Interrupting a thread while http client is running a request might not be the best idea as it might invalidate its underlying resources. I suggest leaving the second fetch running while we propagate the failure (FailedOp) to the downstream operators. But in order to not penalize the next message I suggest increasing the thread pool with a couple more threads, possibly using directly ThreadPoolExecutor <https://devdocs.io/openjdk~8/java/util/concurrent/threadpoolexecutor#ThreadPoolExecutor-int-int-long-java.util.concurrent.TimeUnit-java.util.concurrent.BlockingQueue-java.util.concurrent.ThreadFactory-> with e.g. corePoolSize=2, maximumPoolSize=12, keepAliveTime=5minutes, workQueue=new LinkedBlockedQueue(), threadFactory=the one already created). The flaky test `"an unexpected failure fetching one of the two revisions" should "break the pipeline"` in `GenerateEntityDiffPatchOperationUnitTest` will now have to stop expecting that the second call `getEntityByRevision("Q1", 2)` is always called but also accepts that this call can be omitted (possibly using scalamock noMoreThanOnce <https://scalamock.org/quick-start/>). TASK DETAIL https://phabricator.wikimedia.org/T257795 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: dcausse, Aklapper, CBogen, Akuckartz, darthmon_wmde, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331
_______________________________________________ Wikidata-bugs mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
