jihaozh commented on a change in pull request #4988: [TE] Support weekly
anomaly detection
URL: https://github.com/apache/incubator-pinot/pull/4988#discussion_r367153559
##########
File path:
thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/detection/DetectionUtils.java
##########
@@ -288,4 +288,39 @@ public static Period periodFromTimeUnit(int size,
TimeUnit unit) {
return new Period(TimeUnit.MILLISECONDS.convert(size, unit));
}
}
+
+ /**
+ * Aggregate the time series data frame's value to specified granularity
+ * @param df the data frame
+ * @param origin the aggregation origin time stamp
+ * @param granularityPeriod the aggregation granularity in period
+ * @return the aggregated time series data frame
+ */
+ public static DataFrame aggregateByPeriod(DataFrame df, long origin, Period
granularityPeriod) {
+ return df.groupByPeriod(df.getLongs(COL_TIME), new DateTime(origin),
granularityPeriod).sum(COL_TIME, COL_VALUE);
+ }
+
+ /**
+ * Check if the aggregation result is complete or not, if not, remove it
from the aggregated result.
+ *
+ * For example, say the weekStart is Monday and current data is available
through Jan 8, Wednesday.
+ * the latest data time stamp will be Jan 8. The latest aggregation start
time stamp should be Jan 6, Monday.
+ * In such case, the latest data point is incomplete and should be filtered.
If the latest data time stamp is
+ * Jan 12, Sunday instead, the data is complete and good to use because the
week's data is complete.
+ *
+ * @param df the aggregated data frame to check
+ * @param latestDataTimeStamp the latest data time stamp
+ * @param bucketTimeGranularity the metric's original granularity
+ * @param aggregationGranularityPeriod the granularity after aggregation
+ * @return the filtered data frame
+ */
+ public static DataFrame checkIncompleteAggregation(DataFrame df, long
latestDataTimeStamp,
Review comment:
Yes. I'll rename it to `filterIncompleteAggregation `
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
