Jackie-Jiang commented on a change in pull request #7906:
URL: https://github.com/apache/pinot/pull/7906#discussion_r770164262
##########
File path:
pinot-spi/src/main/java/org/apache/pinot/spi/config/table/UpsertConfig.java
##########
@@ -53,21 +53,27 @@
@JsonPropertyDescription("Partial update strategies.")
private final Map<String, Strategy> _partialUpsertStrategies;
+ @JsonPropertyDescription("global upsert strategy")
+ private final Strategy _globalUpsertStrategy;
+
@JsonPropertyDescription("Column for upsert comparison, default to time
column")
private final String _comparisonColumn;
@JsonCreator
public UpsertConfig(@JsonProperty(value = "mode", required = true) Mode mode,
@JsonProperty("partialUpsertStrategies") @Nullable Map<String, Strategy>
partialUpsertStrategies,
+ @JsonProperty("globalUpsertStrategy") @Nullable Strategy
globalUpsertStrategy,
@JsonProperty("comparisonColumn") @Nullable String comparisonColumn,
@JsonProperty("hashFunction") @Nullable HashFunction hashFunction) {
Preconditions.checkArgument(mode != null, "Upsert mode must be
configured");
_mode = mode;
if (mode == Mode.PARTIAL) {
_partialUpsertStrategies = partialUpsertStrategies != null ?
partialUpsertStrategies : new HashMap<>();
+ _globalUpsertStrategy = globalUpsertStrategy != null ?
globalUpsertStrategy : Strategy.OVERWRITE;
} else {
_partialUpsertStrategies = null;
+ _globalUpsertStrategy = Strategy.OVERWRITE;
Review comment:
Probably set to `null` for full upsert?
##########
File path:
pinot-common/src/test/java/org/apache/pinot/common/utils/config/TableConfigSerDeTest.java
##########
@@ -251,7 +249,8 @@ public void testSerDe()
{
// with upsert config
UpsertConfig upsertConfig =
- new UpsertConfig(UpsertConfig.Mode.FULL, null, "comparison",
UpsertConfig.HashFunction.NONE);
+ new UpsertConfig(UpsertConfig.Mode.FULL, null,
UpsertConfig.Strategy.OVERWRITE, "comparison",
Review comment:
For `FULL` upsert, pass in `null` as the default strategy as that does
not apply? Same for other places
##########
File path:
pinot-spi/src/main/java/org/apache/pinot/spi/config/table/UpsertConfig.java
##########
@@ -53,21 +53,27 @@
@JsonPropertyDescription("Partial update strategies.")
private final Map<String, Strategy> _partialUpsertStrategies;
+ @JsonPropertyDescription("global upsert strategy")
+ private final Strategy _globalUpsertStrategy;
+
@JsonPropertyDescription("Column for upsert comparison, default to time
column")
private final String _comparisonColumn;
@JsonCreator
public UpsertConfig(@JsonProperty(value = "mode", required = true) Mode mode,
@JsonProperty("partialUpsertStrategies") @Nullable Map<String, Strategy>
partialUpsertStrategies,
+ @JsonProperty("globalUpsertStrategy") @Nullable Strategy
globalUpsertStrategy,
@JsonProperty("comparisonColumn") @Nullable String comparisonColumn,
@JsonProperty("hashFunction") @Nullable HashFunction hashFunction) {
Preconditions.checkArgument(mode != null, "Upsert mode must be
configured");
_mode = mode;
if (mode == Mode.PARTIAL) {
_partialUpsertStrategies = partialUpsertStrategies != null ?
partialUpsertStrategies : new HashMap<>();
+ _globalUpsertStrategy = globalUpsertStrategy != null ?
globalUpsertStrategy : Strategy.OVERWRITE;
Review comment:
This is changing the existing behavior. I do agree `OVERWRITE` makes
more sense to partial upsert, but not sure if we want to introduce backward
incompatibility here
##########
File path:
pinot-spi/src/main/java/org/apache/pinot/spi/config/table/UpsertConfig.java
##########
@@ -53,21 +53,27 @@
@JsonPropertyDescription("Partial update strategies.")
private final Map<String, Strategy> _partialUpsertStrategies;
+ @JsonPropertyDescription("global upsert strategy")
+ private final Strategy _globalUpsertStrategy;
Review comment:
Probably more clear if rename to `_defaultPartialUpsertStrategy`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
