[GitHub] [incubator-pinot] jihaozh commented on a change in pull request #4405: [TE] add a thread pool to run preview tasks
jihaozh commented on a change in pull request #4405: [TE] add a thread pool to
run preview tasks
URL: https://github.com/apache/incubator-pinot/pull/4405#discussion_r301336943
##
File path:
thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/detection/yaml/YamlResource.java
##
@@ -724,13 +735,18 @@ private Response runPreview(long start, long end,
Preconditions.checkNotNull(detectionConfig);
DetectionPipeline pipeline = this.loader.from(this.provider,
detectionConfig, start, end);
- result = pipeline.run();
-
+ future = this.executor.submit(pipeline::run);
+ result = future.get(PREVIEW_TIMEOUT, TimeUnit.MILLISECONDS);
} catch (IllegalArgumentException e) {
return processBadRequestResponse(YamlOperations.PREVIEW.name(),
YamlOperations.RUNNING.name(), payload, e);
} catch (InvocationTargetException e) {
responseMessage.put("message", "Failed to run the preview due to " +
e.getTargetException().getMessage());
return Response.serverError().entity(responseMessage).build();
+} catch (TimeoutException e) {
+ // stop the preview
+ future.cancel(true);
Review comment:
good idea. updated.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] jihaozh commented on a change in pull request #4405: [TE] add a thread pool to run preview tasks
jihaozh commented on a change in pull request #4405: [TE] add a thread pool to
run preview tasks
URL: https://github.com/apache/incubator-pinot/pull/4405#discussion_r301334369
##
File path:
thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/detection/yaml/YamlResource.java
##
@@ -145,6 +154,7 @@ public YamlResource() {
this.evaluationDAO = DAORegistry.getInstance().getEvaluationManager();
this.sessionDAO = DAORegistry.getInstance().getSessionDAO();
this.yaml = new Yaml();
+this.executor = Executors.newFixedThreadPool(PREVIEW_PARALLELISM);
Review comment:
Actually, in this case, we want to use a `newFixedThreadPool`, because we
would like to control the maximum number of preview tasks to keep it from
eating all dashboard resource.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] jihaozh commented on a change in pull request #4405: [TE] add a thread pool to run preview tasks
jihaozh commented on a change in pull request #4405: [TE] add a thread pool to run preview tasks URL: https://github.com/apache/incubator-pinot/pull/4405#discussion_r301334391 ## File path: thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/detection/yaml/YamlResource.java ## @@ -115,6 +119,10 @@ private static final String PROP_SESSION_KEY = "sessionKey"; private static final String PROP_PRINCIPAL_TYPE = "principalType"; private static final String PROP_SERVICE = "SERVICE"; + // 5 detection previews are running at the same time at most + private static final int PREVIEW_PARALLELISM = 5; + // max time allowed for a preview task + private static final long PREVIEW_TIMEOUT = TimeUnit.MINUTES.toMillis(5); Review comment: sure This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
