dongxiaoman commented on pull request #7243:
URL: https://github.com/apache/pinot/pull/7243#issuecomment-897006542
@Jackie-Jiang @jackjlli I have updated the code to fix the following:
1. As Jackie pointed out, there are cases when externalView is not updated
yet but the SegmentSelector has picked up. We have very good test coverage and
it help me found out all those failed cases.
2. To support the above cases, introduced `SelectedSegment` class with
sorted Sha256 hash so we can quickly compare whether the Set of segments are a
match. It helps our cases, and should benefit most Offline table or real time
table are not so dynamically updated.
After the optimization, we have either 8x performance improvements, or 2x if
the table is dynamic. The 2x improvement comes from avoiding the frequent
`MurmurHash.getPartition('a7df')` call.
Please help review again, thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
