raghavyadav01 commented on code in PR #19040:
URL: https://github.com/apache/pinot/pull/19040#discussion_r3659796502
##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/openstruct/MutableOpenStructDataSource.java:
##########
@@ -62,10 +63,13 @@ public ComplexFieldSpec getFieldSpec() {
@Override
@Nullable
public DataSource getDataSource(String key) {
- Map<IndexType, IndexReader> indexes = _index.getIndexes(key);
- if (indexes == null || indexes.isEmpty()) {
+ MutableKeyColumn col = _index.getKeyColumn(key);
+ if (col == null) {
return null;
}
+ Map<IndexType, IndexReader> indexes = new
HashMap<>(_index.getIndexes(key));
+ indexes.put(StandardIndexes.nullValueVector(),
+ new PresenceBasedNullValueVector(col.getPresenceBitmap(), _numDocs));
Review Comment:
The mutable path synthesizes a `PresenceBasedNullValueVector` here so
per-key null info is exposed. On the sealed side, `ImmutableSegmentImpl` builds
each materialized key as a plain `ImmutableDataSource`, so
`getNullValueVector()` returns only what was written at seal time — there's no
equivalent synthesis. For a **dense key that's materialized but present in only
some docs**, does the segment build write a null vector for the absent docs? If
not, on immutable segments `buildPerKeyFilterOperator`'s `nullReader == null`
branch would treat those docs as non-null (IS_NULL → Empty, IS_NOT_NULL →
MatchAll), and `hasNullValues` would return false and let a non-scan
aggregation ignore them — both wrong. Worth a test with a partially-present
dense key that survives seal, run on the immutable segment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
