-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35322/
-----------------------------------------------------------
Review request for Ambari, John Speidel, Mahadev Konar, Robert Levas, Sumit
Mohanty, and Yusaku Sako.
Bugs: AMBARI-11850
https://issues.apache.org/jira/browse/AMBARI-11850
Repository: ambari
Description
-------
This patch resolves the performance issues detailed in AMBARI-11850.
The Blueprint export process was taking a long time, due to the fact that the
PropertyProvider instances associated with Ambari Metrics and Alerting always
execute for the Cluster REST resource. In the case of a Blueprint export, the
Cluster resource is accessed with a different format, and a different renderer
that creates the Blueprint based on the Cluster resource's data. The Blueprint
process does not require the Metrics or Alerting data, so the extra processing
time is wasted for this particular case. For very large clusters, this process
can take over 30 minutes or more.
The performance problems have been resolved by the following changes:
1. Added a new method to the Renderer interface, so that custom renderer
implementations can indicate if the property provider support is needed.
2. The BaseRenderer's implementation of this method always returns true, to
keep the behavior for the rest of the renderers as it currently is, with the
exception of the ClusterBlueprintRenderer. In the future, other renderers may
be modified to take advantage of this optimization, depending upon the
functionality required.
3. The ClusterBlueprintRenderer's implementation of this method always returns
false. This indicates that the Blueprint renderer does not require the
additional data provided by AMS or Alerting during a Blueprint export.
Previously, the processing of this data was happening regardless of the
Renderer type.
4. Updated the QueryImpl class to skip the call to
ClusterController.populateResources() if the Renderer does not require property
provider supoprt. This check occurs for resources and any associated
sub-resources.
5. Adds unit tests to verify this change.
Diffs
-----
ambari-server/src/main/java/org/apache/ambari/server/api/query/QueryImpl.java
2319683
ambari-server/src/main/java/org/apache/ambari/server/api/query/render/BaseRenderer.java
7866aa4
ambari-server/src/main/java/org/apache/ambari/server/api/query/render/ClusterBlueprintRenderer.java
cfc9bc0
ambari-server/src/main/java/org/apache/ambari/server/api/query/render/Renderer.java
f353d53
ambari-server/src/test/java/org/apache/ambari/server/api/query/QueryImplTest.java
01361d2
ambari-server/src/test/java/org/apache/ambari/server/api/query/render/ClusterBlueprintRendererTest.java
96abb8c
ambari-server/src/test/java/org/apache/ambari/server/api/query/render/DefaultRendererTest.java
4981a67
Diff: https://reviews.apache.org/r/35322/diff/
Testing
-------
1. Ran the ambari-server unit tests (all passing).
2. Ran basic timing tests against a Blueprint export with this patch applied.
Previously, the export for a 3-node cluster was taking between 7-10 seconds.
With this new patch applied, the Blueprint export takes roughly 50 milliseconds
or less on average. I used a timed "curl" command to obtain this data.
3. Took the Blueprint exported from the test in #2, and used this Blueprint to
deploy a brand new cluster, to verify that the Blueprint was valid. The
cluster deployment succeeded. I had to make a slight change to the Blueprint,
to add command retry configuration, but this is un-related to the performance
changes. The exported Blueprint will POST correctly without any manual
changes. The cluster-env changes are just to active the new command retry
feature for multi-node Blueprint deployments.
Thanks,
Robert Nettleton
