walterddr commented on issue #10829:
URL: https://github.com/apache/pinot/issues/10829#issuecomment-1581239590
found another race-condition -->
1. when pipeline breaker returns it is side-cart loading the result map
materialized from the PipelineBreakerOperator --> which returns immediately
after the mailbox receives the final block.
2. however, while the mailbox received the final block, it will return the
EOS back to the opChainScheduler, then OpChainScheduler deregister/close the
opChain properly.
- once the opChain close properly, the leaf stage can then be registered
if the opChain was closed after the leaf stage opChain being registered, it
throws:
```
2023-06-07T01:27:12.8609939Z [INFO] Running
org.apache.pinot.query.runtime.queries.ResourceBasedQueriesTest
2023-06-07T01:27:28.3251491Z 01:27:28.203 ERROR [QueryRunner]
[QueryServerTest_Server-2-thread-3] Error executing leaf stage for:
-7077425705764084110:2
2023-06-07T01:27:28.3255009Z java.lang.IllegalStateException: Tried to
re-register op-chain: -7077425705764084110_0_2
2023-06-07T01:27:28.3255606Z at
com.google.common.base.Preconditions.checkState(Preconditions.java:510)
~[guava-30.1.1-jre.jar:?]
2023-06-07T01:27:28.3256318Z at
org.apache.pinot.query.runtime.executor.RoundRobinScheduler.register(RoundRobinScheduler.java:146)
~[classes/:?]
2023-06-07T01:27:28.3257063Z at
org.apache.pinot.query.runtime.executor.OpChainSchedulerService.register(OpChainSchedulerService.java:135)
~[classes/:?]
2023-06-07T01:27:28.3257752Z at
org.apache.pinot.query.runtime.QueryRunner.processQuery(QueryRunner.java:167)
~[classes/:?]
2023-06-07T01:27:28.3258481Z at
org.apache.pinot.query.QueryServerEnclosure.lambda$processQuery$1(QueryServerEnclosure.java:135)
~[test-classes/:?]
2023-06-07T01:27:28.3259318Z at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?]
2023-06-07T01:27:28.3259904Z at
java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
2023-06-07T01:27:28.3260431Z at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
[?:?]
2023-06-07T01:27:28.3260991Z at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
[?:?]
2023-06-07T01:27:28.3261444Z at
java.lang.Thread.run(Thread.java:829) [?:?]
2023-06-07T01:27:28.3262254Z 01:27:28.312 ERROR [OpChainSchedulerService]
[query_intermediate_worker_on_41325_port-5-thread-2]
(OpChain{-7077425705764084110_0_1}): Failed to execute operator chain!
(-7077425705764084110_0_1) Queued Count: 3, Executing Time: 0ms, Queued Time:
128ms
2023-06-07T01:27:28.3262881Z java.lang.InterruptedException: Query was
cancelled!
2023-06-07T01:27:28.3263472Z at
org.apache.pinot.query.runtime.executor.OpChainSchedulerService$1.runJob(OpChainSchedulerService.java:88)
[classes/:?]
2023-06-07T01:27:28.3264560Z at
org.apache.pinot.core.util.trace.TraceRunnable.run(TraceRunnable.java:40)
[pinot-core-0.13.0-SNAPSHOT.jar:0.13.0-SNAPSHOT-3b2e31cfad6d327ca41021e091812f0f7e17e453]
2023-06-07T01:27:28.3265217Z at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?]
2023-06-07T01:27:28.3265689Z at
java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
2023-06-07T01:27:28.3266195Z at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
[?:?]
2023-06-07T01:27:28.3266738Z at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
[?:?]
2023-06-07T01:27:28.3267166Z at
java.lang.Thread.run(Thread.java:829) [?:?]
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
