[
https://issues.apache.org/jira/browse/AMBARI-16131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15261014#comment-15261014
]
Hudson commented on AMBARI-16131:
---------------------------------
SUCCESS: Integrated in Ambari-trunk-Commit #4750 (See
[https://builds.apache.org/job/Ambari-trunk-Commit/4750/])
AMBARI-16131 - Prevent Views From Causing a Loss of Service For Ambari
(jhurley:
[http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=85ac3c22ea6aedd4a75e79ee25798677e07a7bdd])
*
ambari-server/src/main/java/org/apache/ambari/server/view/ViewThrottleFilter.java
*
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
*
ambari-server/src/test/java/org/apache/ambari/server/view/ViewThrottleFilterTest.java
* ambari-server/src/main/java/org/apache/ambari/server/view/ViewRegistry.java
* ambari-server/src/main/java/org/apache/ambari/server/utils/VersionUtils.java
*
ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
> Prevent Views From Causing a Loss of Service For Ambari
> -------------------------------------------------------
>
> Key: AMBARI-16131
> URL: https://issues.apache.org/jira/browse/AMBARI-16131
> Project: Ambari
> Issue Type: Bug
> Components: ambari-views
> Affects Versions: 2.0.0
> Reporter: Jonathan Hurley
> Assignee: Jonathan Hurley
> Priority: Critical
> Fix For: 2.4.0, 2.2-next
>
> Attachments: AMBARI-16131.patch
>
>
> The underlying problem is that views are accessed off of the REST endpoint
> ({{/api/v1/views}}). This means that the Ambari REST API connector is going
> to handle the request from its own threadpool. There is no way to configure
> Jetty to use a different threadpool for the same connector. As a result, if a
> request to load a view holds the Jetty thread hostage, eventually we will see
> thread starvation and loss of service.
> An example of this situation is a view which makes an innocent request to a
> remote resource. If the view's request has a timeout of 60 seconds, then the
> Jetty thread is going to be held for that amount of time. With concurrent
> users and multiple instances of that view deployed, the Jetty threadpool can
> becomes exhausted quickly.
> Although there are more graceful ways of handling this situation, they mostly
> involve substantial re-architecture and design:
> - The use of a new connector and threadpool would require binding to another
> port for view requests. This will cause problems with "local" views and their
> assumption that if they run on the Ambari server they can share the same
> session.
> - The use of a
> [Continuation|https://wiki.eclipse.org/Jetty/Feature/Continuations] in Jetty
> which can suspend the incoming request. We would need the ability for views
> to signal that they have completed their work in order to proceed with the
> suspended request.
> A quicker and far less invasive fix would be to create a filter which
> intercepts requests for views. It will determine how many executing view
> requests exist and decide if it will allow the new request through. For
> example, if configured to allow a maximum of 10 concurrent view requests,
> then the 11th request would be denied with an {{HTTP 503 - Service
> Unavailable}}. Although the thread is temporarily used while the filter is
> processing, it's quickly returned to the Jetty pool when it's determined
> there are too many other running view requests.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
