yashmayya opened a new pull request, #20104: URL: https://github.com/apache/pinot/pull/20104
Adds `MultiStageBrokerRequestHandler#prepareStageForDispatch`, an empty protected hook. The broker calls it for each stage right before it sends the stage to its workers: - to run the query: after planning, authorization and quota checks, before throttling and dispatch - to explain the query on the servers (`explainAskingServers`): before each stage goes to `QueryDispatcher#explain` A subclass can add entries to the custom properties of the stage (`DispatchablePlanFragment#getCustomProperties()`). The workers of the stage get them in their stage metadata. It follows the existing `prepareCompiledQueryForPlanning` and `getEstimatedNumQueryThreads` hooks. Today `getEstimatedNumQueryThreads` is the only subclass hook that sees the final plan. It exists for throttling and does not see the explain path. **Use case:** a custom worker runtime (an `OpChainConverter` plugin) can need data from the broker for each stage. For example, a runtime with its own mailbox port can get the port of each receiver from the plan. Without the hook, every worker has to read it from ZooKeeper. **Compatibility:** additive only. The default does nothing. **Testing:** `MultiStageBrokerRequestHandlerTest#testPrepareStageForDispatchHook` plans a real query and adds a property to each stage in the hook. It checks that the plan that reaches throttling has the property on all stages, and that each stage sent to `QueryDispatcher#explain` has it. The test fails if either hook call is removed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
