rohityadav1993 opened a new issue, #12331: URL: https://github.com/apache/pinot/issues/12331
There are ways to upload pre-generated segments to Pinot tables. This provides a mechanism for backfills for realtime and offline tables (bootstrapping a new table or correcting data). If upsert metadata is properly provided for full upsert tables, then pre-partitioned segments by the primary key can also be uploaded with correct upsert behaviour. The above approach falls short when uploading segments of partial upsert table as there is an insert time transformation on the row by merging it with previously persisted row for a primary key. Instead of uploading pre-generated segments to partial upsert table, one can also not put the backfill data in the stream for realtime ingestion to take care of insert time transformation (merging) as during backfills one would want to remove all the rows of a primary key persisted and then have a fresh ingestion for the primary key. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
