swaminathanmanish commented on code in PR #14228:
URL: https://github.com/apache/pinot/pull/14228#discussion_r1808636526
##########
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/PinotHelixResourceManager.java:
##########
@@ -1815,6 +1821,28 @@ void validateTableTenantConfig(TableConfig tableConfig) {
}
}
+ @VisibleForTesting
+ void validateTableTaskMinionInstanceTagConfig(TableConfig tableConfig) {
+
+ List<InstanceConfig> allMinionWorkerInstanceConfigs =
getAllMinionInstanceConfigs();
+
+ //extract all minionInstanceTags from allMinionWorkerInstanceConfigs
+ Set<String> minionInstanceTagSet =
allMinionWorkerInstanceConfigs.stream().map(InstanceConfig::getTags)
+ .collect(HashSet::new, Set::addAll, Set::addAll);
+
+ if (tableConfig.getTaskConfig() != null &&
tableConfig.getTaskConfig().getTaskTypeConfigsMap() != null) {
+ tableConfig.getTaskConfig().getTaskTypeConfigsMap().forEach((taskType,
taskTypeConfig) -> {
+ String taskInstanceTag =
taskTypeConfig.getOrDefault(PinotTaskManager.MINION_INSTANCE_TAG_CONFIG,
+ CommonConstants.Helix.UNTAGGED_MINION_INSTANCE);
Review Comment:
Assuming existing integ tests will handle default case when there are no
tags .
##########
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/PinotHelixResourceManager.java:
##########
@@ -1815,6 +1821,28 @@ void validateTableTenantConfig(TableConfig tableConfig) {
}
}
+ @VisibleForTesting
+ void validateTableTaskMinionInstanceTagConfig(TableConfig tableConfig) {
+
+ List<InstanceConfig> allMinionWorkerInstanceConfigs =
getAllMinionInstanceConfigs();
+
+ //extract all minionInstanceTags from allMinionWorkerInstanceConfigs
+ Set<String> minionInstanceTagSet =
allMinionWorkerInstanceConfigs.stream().map(InstanceConfig::getTags)
+ .collect(HashSet::new, Set::addAll, Set::addAll);
+
+ if (tableConfig.getTaskConfig() != null &&
tableConfig.getTaskConfig().getTaskTypeConfigsMap() != null) {
+ tableConfig.getTaskConfig().getTaskTypeConfigsMap().forEach((taskType,
taskTypeConfig) -> {
+ String taskInstanceTag =
taskTypeConfig.getOrDefault(PinotTaskManager.MINION_INSTANCE_TAG_CONFIG,
+ CommonConstants.Helix.UNTAGGED_MINION_INSTANCE);
+ if (!minionInstanceTagSet.contains(taskInstanceTag)) {
+ throw new InvalidTableConfigException(
+ String.format("Failed to find minion instances with tag: %s for
table: %s", taskInstanceTag,
Review Comment:
How will this error surface for adhoc triggers ? (Execute call) where task
params are in the payload .
cc @tibrewalpratik17
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
