Jackie-Jiang commented on code in PR #17550:
URL: https://github.com/apache/pinot/pull/17550#discussion_r2718616368
##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/indexsegment/mutable/MutableSegmentImpl.java:
##########
@@ -640,7 +640,24 @@ public boolean index(GenericRow row, @Nullable
StreamMessageMetadata metadata)
if (isUpsertEnabled()) {
RecordInfo recordInfo = getRecordInfo(row, numDocsIndexed);
GenericRow updatedRow =
_partitionUpsertMetadataManager.updateRecord(row, recordInfo);
+ // NOTE: out-of-order records can not be dropped or marked when
consistent upsert view is enabled.
+ // Since Indexing the record and updation of _numDocsIndexed counter
happens before updating the upsert
+ // metadata, we wouldn't be able to actually drop or mark those records
as dropped. This order is important for
+ // consistent upsert view, otherwise the latest doc can be missed by
query due to 'docId < _numDocs' check
+ // in query filter operators. Here the record becomes queryable before
validDocIds bitmaps are updated.
if (_upsertConsistencyMode != UpsertConfig.ConsistencyMode.NONE) {
+ if (_upsertDropOutOfOrderRecord) {
Review Comment:
Move these checks into `UpsertContext.Builder.build()`. The validations
should be performed there
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
