This is an automated email from the ASF dual-hosted git repository.
gortiz pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pinot.git
The following commit(s) were added to refs/heads/master by this push:
new 85236c4393 Invalidate from cache the stats of opchains that finish
successfully. (#15967)
85236c4393 is described below
commit 85236c439321b26cfbb4ef2ecff6e2c817284de5
Author: Gonzalo Ortiz Jaureguizar <[email protected]>
AuthorDate: Tue Jun 3 09:58:58 2025 +0200
Invalidate from cache the stats of opchains that finish successfully.
(#15967)
We have detected that some large queries may end up consuming too much
memory for too long in the stats cache used to return stats on timeout.
A way to solve that is to remove the stats as soon as the opchain finishes
successfully. This should keep the cache almost empty (given most of the time
opchains don't fail) at the cost of losing the stats of fast stages when other
stage timeouts.
This is not the best solution but one that is very easy to implement. We
will work in better long term solutions.
---
.../org/apache/pinot/query/runtime/executor/OpChainSchedulerService.java | 1 +
1 file changed, 1 insertion(+)
diff --git
a/pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/executor/OpChainSchedulerService.java
b/pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/executor/OpChainSchedulerService.java
index 2ed131b782..18d34f35c6 100644
---
a/pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/executor/OpChainSchedulerService.java
+++
b/pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/executor/OpChainSchedulerService.java
@@ -96,6 +96,7 @@ public class OpChainSchedulerService {
LOGGER.error("({}): Completed erroneously {} {}", operatorChain,
stats, errorBlock.getErrorMessages());
} else {
LOGGER.debug("({}): Completed {}", operatorChain, stats);
+ _opChainCache.invalidate(operatorChain.getId());
}
} catch (Exception e) {
LOGGER.error("({}): Failed to execute operator chain!",
operatorChain, e);
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
