xiaohui-sun commented on a change in pull request #4136: [TE] catch exceptions
in dimension exploration
URL: https://github.com/apache/incubator-pinot/pull/4136#discussion_r276786512
##########
File path:
thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/detection/algorithm/DimensionWrapper.java
##########
@@ -202,10 +202,14 @@ public DetectionPipelineResult run() throws Exception {
for (MetricEntity metric : nestedMetrics) {
for (Map<String, Object> properties : this.nestedProperties) {
LOG.info("running detection for {}", metric.toString());
- DetectionPipelineResult intermediate = this.runNested(metric,
properties);
- lastTimeStamps.add(intermediate.getLastTimestamp());
- anomalies.addAll(intermediate.getAnomalies());
- diagnostics.put(metric.getUrn(), intermediate.getDiagnostics());
+ try {
+ DetectionPipelineResult intermediate = this.runNested(metric,
properties);
+ lastTimeStamps.add(intermediate.getLastTimestamp());
+ anomalies.addAll(intermediate.getAnomalies());
+ diagnostics.put(metric.getUrn(), intermediate.getDiagnostics());
+ } catch (Exception e) {
Review comment:
It is good we raised this issue.
I suggest to do the following to make it more effective, like what we did
for detection flow:
1. Keep track of exceptions for each metric.
2. If it failed for all of the first 10 metrics then "exit early", and then
throw exception.
3. If it failed for all metrics then throw exception.
After that, modify to send exception out in detection flow.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
