xiaohui-sun commented on a change in pull request #4505: [TE] Update
AnomalyFlattenResource
URL: https://github.com/apache/incubator-pinot/pull/4505#discussion_r311870392
##########
File path:
thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/dashboard/resources/AnomalyFlattenResource.java
##########
@@ -44,66 +57,140 @@
@Path("thirdeye/table")
@Api(tags = {Constants.ANOMALY_TAG})
public class AnomalyFlattenResource {
- private MergedAnomalyResultManager mergedAnomalyResultDAO;
+ private final MergedAnomalyResultManager mergedAnomalyResultDAO;
+ private final MetricConfigManager metricConfigDAO;
+ private final DatasetConfigManager datasetConfgDAO;
public static final String ANOMALY_ID = "anomalyId";
public static final String ANOMALY_COMMENT = "comment";
+ private static final String DATAFRAME_VALUE = "value";
+ private static final String DEFAULT_COMMENT_FORMAT = "#%d is %s";
- public AnomalyFlattenResource(MergedAnomalyResultManager
mergedAnomalyResultDAO) {
+ public AnomalyFlattenResource(MergedAnomalyResultManager
mergedAnomalyResultDAO, DatasetConfigManager datasetConfgDAO,
+ MetricConfigManager metricConfigDAO) {
this.mergedAnomalyResultDAO = mergedAnomalyResultDAO;
+ this.datasetConfgDAO = datasetConfgDAO;
+ this.metricConfigDAO = metricConfigDAO;
}
+
/**
- * Returns a list of formatted merged anomalies for UI to render a table
- * @param detectionConfigId detectionConfigId
+ * Returns a list of formatted metric values and anomaly comments for UI to
generate a table
+ * @param metricIdStr a string of metric ids separated by comma
* @param start start time in epoc milliseconds
* @param end end time in epoc milliseconds
- * @param dimensionKeys a list of keys in dimensions; if null, will return
all dimension keys for the anomaly
- * @return a list of formatted anomalies
+ * @param dimensionStrings a list of keys in dimensions joined by comma
+ * @return a list of formatted metric info and anomaly comments
*/
@GET
- @ApiOperation(value = "Returns a list of formatted merged anomalies ")
- public List<Map<String, Object>> flatAnomalyResults(
- @ApiParam("detection config id") @QueryParam("detectionConfigId") long
detectionConfigId,
+ @ApiOperation(value = "View a flatted merged anomalies for collection")
+ @Produces("Application/json")
+ public List<Map<String, Object>> listDimensionValues(
+ @ApiParam("metric config id") @NotNull @QueryParam("metricIds") String
metricIdStr,
@ApiParam("start time for anomalies") @QueryParam("start") long start,
@ApiParam("end time for anomalies") @QueryParam("end") long end,
- @ApiParam("dimension keys") @QueryParam("dimensionKeys") List<String>
dimensionKeys) {
- // Retrieve anomalies
- List<MergedAnomalyResultDTO> anomalies = mergedAnomalyResultDAO.
- findByStartTimeInRangeAndDetectionConfigId(start, end,
detectionConfigId);
+ @ApiParam("dimension keys") @NotNull @QueryParam("dimensionKeys") String
dimensionStrings) throws Exception {
+ Preconditions.checkArgument(StringUtils.isNotBlank(metricIdStr));
+ List<String> dimensionKeys = Arrays.asList(dimensionStrings.split(","));
+ List<MergedAnomalyResultDTO> anomalies = new ArrayList<>();
+ List<MetricConfigDTO> metrics = new ArrayList<>();
+ for (String metricId : metricIdStr.split(",")) {
+ long id = Long.valueOf(metricId);
+ metrics.add(metricConfigDAO.findById(id));
+
anomalies.addAll(mergedAnomalyResultDAO.findAnomaliesByMetricIdAndTimeRange(id,
start, end));
+ }
- // flatten anomaly result information
- List<Map<String, Object>> resultList = new ArrayList<>();
- for (MergedAnomalyResultDTO result : anomalies) {
- resultList.add(flatAnomalyResult(result, dimensionKeys));
+ Map<String, DataFrame> metricDataFrame = new HashMap<>();
+
+ AggregationLoader aggregationLoader =
+ new DefaultAggregationLoader(metricConfigDAO, datasetConfgDAO,
+ ThirdEyeCacheRegistry.getInstance().getQueryCache(),
+ ThirdEyeCacheRegistry.getInstance().getDatasetMaxDataTimeCache());
+
+ for (MetricConfigDTO metricDTO : metrics) {
+ MetricSlice metricSlice = MetricSlice.from(metricDTO.getId(), start,
end);
+ DataFrame dataFrame = aggregationLoader.loadAggregate(metricSlice,
dimensionKeys, -1);
Review comment:
This may take a long/unpredicted time.
Suggest to check this PR to add a thread pool and timeout.
https://github.com/apache/incubator-pinot/pull/4405
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
