jamesyfshao opened a new pull request #5175: add pinot upsert features to pinot common URL: https://github.com/apache/incubator-pinot/pull/5175 This diff will create necessary pinot schema/constant value changes to support pinot upsert features. The following will be changed: 1. add updateSchematic field to schema and table definition: this value can be either "upsert" or "append". When it is "upsert" pinot server/broker will apply proper upsert logic to ensure that table is updating based on primary key. a few notes: - I am planning to simplify this config to be a boolean value, but due to several of our tables are already using this config, I will need to spend some time to configure out a way to migrate this properly. - Currently, both schema & table requires this config to ensure that when we create table/schema they will have easy way to check if they are for upsert and ensure proper checks are enforced. 2. add primary key config: this value will point to one of the column in this table as the primary key of this upsert table. Only matter if the table is for upsert 3. add offset key config: this value will point to one of the column in this table to store the offset of this table. Only matter if the table is for upsert
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
