klsince commented on code in PR #11811:
URL: https://github.com/apache/pinot/pull/11811#discussion_r1361002916
##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/indexsegment/mutable/MutableSegmentImpl.java:
##########
@@ -489,12 +489,17 @@ public boolean index(GenericRow row, @Nullable
RowMetadata rowMetadata)
if (isUpsertEnabled()) {
RecordInfo recordInfo = getRecordInfo(row, numDocsIndexed);
GenericRow updatedRow =
_partitionUpsertMetadataManager.updateRecord(row, recordInfo);
- updateDictionary(updatedRow);
- addNewRow(numDocsIndexed, updatedRow);
- // Update number of documents indexed before handling the upsert
metadata so that the record becomes queryable
- // once validated
- canTakeMore = numDocsIndexed++ < _capacity;
- _partitionUpsertMetadataManager.addRecord(this, recordInfo);
+ // if record doesn't need to be dropped, then persist in segment and
update metadata hashmap
Review Comment:
nit:
```
if (shouldDropRecord...) {
numDocsIndexed++
}
canTakeMore = numDocsIndexed < _capacity;
```
##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/upsert/BasePartitionUpsertMetadataManager.java:
##########
@@ -535,6 +537,13 @@ public GenericRow updateRecord(GenericRow record,
RecordInfo recordInfo) {
protected abstract GenericRow doUpdateRecord(GenericRow record, RecordInfo
recordInfo);
+ @Override
+ public boolean shouldDropRecord(RecordInfo recordInfo) {
+ return doShouldDropRecord(recordInfo);
Review Comment:
doShouldDropRecord() seems not necessary, as shouldDropRecord() does nothing
but call this method.
how about to move the changes from L307-L326 in
`ConcurrentMapPartitionUpsertMetadataManager` to this shouldDropRecord() method
and add a `handleOutOfOrderEvent` method in base class to let subclass to
override this part only.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
