siddharthteotia edited a comment on pull request #7898: URL: https://github.com/apache/pinot/pull/7898#issuecomment-993108389
I don't fully understand the optimization here. Looks like the new flow will be `ImmutableSegmentLoader` will call `SegmentPReprocessor.needsPreprocess()` and if this returns true, will the segment be pre-processed by the server during load. However, the code is using segment directory interface which seems to imply that segment exists on the server unless this is being done specifically for a custom/plugin implementation different from SegmentLocalFSDirectory ? In that case I can see the point of just looking at the metadata (whether the column has index or not, whether column exists or not etc) without necessarily having the entire segment payload on the server before making the decision -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
