somandal opened a new pull request, #9740: URL: https://github.com/apache/pinot/pull/9740
Related OSS Issue: https://github.com/apache/pinot/issues/6473 Related OSS PRs: https://github.com/apache/pinot/pull/9333 Document outlining the breakup of PRs and the approach to solve various issues in this space: https://docs.google.com/document/d/1MNLLhYCg5e-UFBQ6wTBODd41sDsbjevwRfwoGuNowWw/edit?usp=sharing This PR adds a feature to disable the forward index for existing columns via the reload path. A new method has been added to the `IndexHandler` called `postUpdateIndicesCleanup` which performs cleanup operations for the index handlers. For now this is going to be used only by the `ForwardIndexHandler` to cleanup the forward index after all other `IndexHandlers` have run. The forward index may be required by other `IndexHandlers` to construct their own indices due to which the forward index should only be cleaned up after all these have run. In the future, when support is added to enable the forward index on a column where it has been disabled we will need to create a temporary ad-hoc forward index just to create other indices and then clean it up afterwards. This new method will help assist with this as well. TODO: Working on adding more tests, especially on the query path This PR does not add support for the following (and there will be follow up PRs to handle these): - Enabling the forward index for existing columns which have it disabled - Disabling the forward index for new columns that are added (default column handling) for MV columns - Dealing with duplicated entries within a multi-value row - Derived column handling -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
