somandal opened a new pull request, #10570: URL: https://github.com/apache/pinot/pull/10570
This PR addresses the issue found: https://github.com/apache/pinot/issues/10555 Summary: The logic added as part of PR https://github.com/apache/pinot/pull/10408 added support to order as part of the `MailboxReceiveOperator` which essentially stored all the rows in a `PriorityQueue` instead of returning them right away. Due to this we got into a situation where the blocks were added to the `PriorityQueue` but we never re-polled immediately to check for `EOS` but returned a no-op block. Due to this, since no new blocks were sent the `_seenMail` was never set and the operator had to be reawakened after the default timeout. Fix: This PR fixes the above by splitting the MailboxReceiveOperator into a `MailboxReceiveOperator` and a `SortedMailboxReceiveOperator`. The sorted version keeps polling for data from all the mailboxes until it gets only 'null' blocks or gets EOS from all mailboxes. The `MailboxReceiveOperator` does not perform any sorting and will behave the same way it did prior to the PR https://github.com/apache/pinot/pull/10408. Testing done: - Unit tests in planning and runtime modules - MultiStageEngineIntegrationTest - to ensure that this fixes the increase in runtime from 30s to 500s (and fixes the regression in ORDER BY queries) cc @walterddr @ankitsultana @siddharthteotia @vvivekiyer -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
