This is an automatically generated e-mail. To reply, visit:

(Updated July 1, 2016, 4:25 p.m.)

Review request for Ambari, Jonathan Hurley, Mahadev Konar, Oliver Szabo, and 
Sumit Mohanty.


Updated patch to include review feedback.

Bugs: AMBARI-17510

Repository: ambari


This patch implements some updates to the Ambari LogSearch integration layer in 
order to resolve AMBARI-17510. 

The initial implementation of the LoggingSearchPropertyProvider directly made 
calls to the LogSearch Server's REST endpoint, in order to obtain LogSearch 
metadata, and to attach this information to the Ambari HostComponent REST 
resource.  Recent testing has shown that there are some deployment scenarios 
(larger clusters, rolling upgrade) that demonstrate a performance problem in 
the LogSearch integration code.  

This patch addresses this problem by implementing the following:

1. A new, injectable service 
(org.apache.ambari.server.controller.logging.LogSearchDataRetrievalService) has 
been introduced into the LoggingSearchPropertyProvider.  The property provider 
now consults this service to obtain the required LogSearch data.  This new 
service is based on Jonathan Hurley's work in 
2. The LogSearchDataRetrievalService utilizes an internal cache to maintain a 
map of the LogSearch data, which typically will not change.  If the cache does 
not contain this information for a given HostComponent, the retrieval service 
will make the required REST call to the LogSearch service to obtain this data, 
and then add this to the cache.  The remote REST call now happens on a separate 
thread from the Ambari REST request handler, which removes the possibility of 
causing degradation on the Ambari REST endpoing. 
3. The LoggingRequestHelperImpl class has been slightly updated, in order to 
add 5 second connect/read timeouts on the LogSearch requests.  This should 
prevent any threads from hanging infinitely if the LogSearch server is slow to 
respond.  In a future patch, we might want to make these timeouts configurable.

Diffs (updated)


Diff: https://reviews.apache.org/r/49474/diff/


1. Ran the ambari-server "mvn clean test" suite, and all Java and Python unit 
tests passed with this change applied.
2. Deployed a 3-node cluster with HDFS, Yarn, and LogSearch selected with this 
patch applied. Verified that the cluster deployed successfully, and that the 
"host_components" REST resource (the resource associated with this performance 
problem) returns properly.  Ran 6-7 separate REST clients making concurrent 
requests on the "host_componets" resource in Ambari for over 40 minutes, and 
verified that there is no noticeable slowdown.  Verified that ambari-server 
remains responsive after this test. Also verified via DEBUG-level logging that 
the LogSearch remote REST requests are indeed happening on a separate thread 
3. Deployed a 3-node cluster with HDFS and Yarn selected (LogSearch not 
selected) with this patch applied.  Verified that the cluster deploys properly, 
and that the Ambari REST "host_components" resource returns properly when 
LogSearch is not included in the cluster.


Robert Nettleton

Reply via email to