somandal commented on PR #9333:
URL: https://github.com/apache/pinot/pull/9333#issuecomment-1241174482
> We should allow disabling forward index for sorted column, which is a
no-op because the inverted index for sorted column can be used as forward
index. I believe it is already allowed in the implementation because the
default column is always sorted. No need to add this in the validation.
Thanks for the suggestion @Jackie-Jiang. From going through the code it
looks like for sorted columns the forward index and inverted index are
essentially the same. Code snippet from the `PhysicalColumnIndexCreator`:
```
// Single-value
if (metadata.isSorted()) {
// Sorted
// forwardIndexDisabled columns do not need to be handled here as
forward index cannot be disabled on a
// sorted column
SortedIndexReader<?> sortedIndexReader =
indexReaderProvider.newSortedIndexReader(fwdIndexBuffer, metadata);
_forwardIndex = sortedIndexReader;
_invertedIndex = sortedIndexReader;
_fstIndex = null;
return;
}
```
I also see code on the segment creation path where we skip creating the
inverted index if the column is sorted even if the inverted index is enabled
for the column. So keeping these in mind, we didn't think it made much sense to
disable forward index for such columns.
Let me know your thoughts based on the above. cc @siddharthteotia
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
