jihaozh commented on a change in pull request #5618:
URL: https://github.com/apache/incubator-pinot/pull/5618#discussion_r446346639
##########
File path:
thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/auto/onboard/AutoOnboardPinotMetadataSource.java
##########
@@ -200,23 +201,26 @@ private void addNewDataset(String dataset, Schema schema,
Map<String, String> cu
/**
* Refreshes an existing dataset in the thirdeye database
* with any dimension/metric changes from pinot schema
- * @param dataset
- * @param schema
- * @param datasetConfig
*/
- private void refreshOldDataset(String dataset, Schema schema, Map<String,
String> customConfigs,
- DatasetConfigDTO datasetConfig) throws Exception {
- checkDimensionChanges(dataset, datasetConfig, schema);
+ private void refreshOldDataset(String dataset, Schema schema, String
timeColumnName,
+ Map<String, String> customConfigs, DatasetConfigDTO datasetConfig) {
+ checkDimensionChanges(dataset, datasetConfig, schema, timeColumnName);
checkMetricChanges(dataset, datasetConfig, schema);
- checkTimeFieldChanges(datasetConfig, schema);
+ checkTimeFieldChanges(datasetConfig, schema, timeColumnName);
appendNewCustomConfigs(datasetConfig, customConfigs);
checkNonAdditive(datasetConfig);
datasetConfig.setActive(true);
}
- private void checkDimensionChanges(String dataset, DatasetConfigDTO
datasetConfig, Schema schema) {
+ private void checkDimensionChanges(String dataset, DatasetConfigDTO
datasetConfig, Schema schema,
+ String timeColumnName) {
LOG.info("Checking for dimensions changes in {}", dataset);
- List<String> schemaDimensions = schema.getDimensionNames();
+ List<String> schemaDimensions = new
ArrayList<>(schema.getDimensionNames());
+ for (String dateTimeColumn : schema.getDateTimeNames()) { // treat all
dateTimeFields specs as dimensions, except the primary time column
Review comment:
Thanks. I think we can ignore the remaining time columns for now. In the
future, we can think about how to support those in TE.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
