Jackie-Jiang commented on code in PR #11811:
URL: https://github.com/apache/pinot/pull/11811#discussion_r1362784822
##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/indexsegment/mutable/MutableSegmentImpl.java:
##########
@@ -489,12 +489,16 @@ public boolean index(GenericRow row, @Nullable
RowMetadata rowMetadata)
if (isUpsertEnabled()) {
RecordInfo recordInfo = getRecordInfo(row, numDocsIndexed);
GenericRow updatedRow =
_partitionUpsertMetadataManager.updateRecord(row, recordInfo);
- updateDictionary(updatedRow);
- addNewRow(numDocsIndexed, updatedRow);
- // Update number of documents indexed before handling the upsert
metadata so that the record becomes queryable
- // once validated
- canTakeMore = numDocsIndexed++ < _capacity;
- _partitionUpsertMetadataManager.addRecord(this, recordInfo);
+ // if record doesn't need to be dropped, then persist in segment and
update metadata hashmap
+ if (!_partitionUpsertMetadataManager.shouldDropRecord(recordInfo)) {
Review Comment:
The analysis is correct. What I meant is that currently it is anyway not
handled correctly, and we should solve this problem in a separate PR. Assuming
`addNewRow()` failed, dictionary was already updated, and the first several
columns might also be added already. Currently it works because that call never
fails. The proper fix should be to ensure even if it throws exception, we
should continue on other columns (e.g. put a default value for the failed
column), and increase the doc id anyway.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
