Jackie-Jiang commented on issue #12396: URL: https://github.com/apache/pinot/issues/12396#issuecomment-1942855800
Thanks for summarizing the issues! For the first 2 issues, we need to figure out how to handle partial-upsert with same comparison value. For partial upsert, the insert order is critical, and we need to design a way to tell which is the final valid record. One way I can think of is to add an extra virtual column to track the version of the partial-upsert record for the tie comparison value. Within each segment, the version starts from 0, and whenever a new record has tie comparison value, we increment it by one. When replacing the committed segment, we can use this version number to break tie. Another similar approach is to record the original doc id for each record. The drawback is that it might take more storage because all values will be unique. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
