-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39536/
-----------------------------------------------------------
Review request for Ambari, Alejandro Fernandez, Dmytro Sen, Jonathan Hurley,
and Sumit Mohanty.
Bugs: AMBARI-13517
https://issues.apache.org/jira/browse/AMBARI-13517
Repository: ambari
Description
-------
- Max Heap restriction problem: Ehcache library that we use has a limit of how
far in the object graph it traverses to find size of an Cached reference, this
is to make it performant. Default limit = 1000.
- User gets following Warning which are harmless unless the data cannot fit in
memory and eviction does not kick in:
WARN [qtp-client-70] ObjectGraphWalker:209 - The configured limit of 1,000
object references was reached while attempting to calculate the size of the
object graph. Severe performance degradation could occur
if the sizing operation continues. This can be avoided by setting the
CacheManger or Cache <sizeOfPolicy> elements maxDepthExceededBehavior to
"abort" or adding stop points with @IgnoreSizeOf annotations. If performance
degradation
is NOT an issue at the configured limit, raise the limit value using the
CacheManager or Cache <sizeOfPolicy> elements maxDepth attribute. For more
information, see the Ehcache configuration documentation.
_Objective of the patch_:
- Provide a custom Sizing engine for Ehcache that will provide a close
approximation of the data in the cache with significant perf gain. Premise:
Since we know the data structures we can make better estimates.
- Expectation from sizing engine : Discrepancy of less than 10K for a DS of
size 10 MB (Proven with a unit test)
Diffs
-----
ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/TimelineMetric.java
8b8df06
ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/cache/TimelineMetricsCache.java
224b490
ambari-metrics/ambari-metrics-common/src/test/java/org/apache/hadoop/metrics2/sink/timeline/cache/TimelineMetricsCacheTest.java
4a13d63
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java
9c0b94d
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java
9488316
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricReadHelper.java
573e09d
ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/ITClusterAggregator.java
b7b1737
ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/ITMetricAggregator.java
a3640d0
ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/MetricTestHelper.java
26771d7
ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/MetricsPaddingMethod.java
522c0bd
ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineMetricCacheProvider.java
6d80687
ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineMetricsCacheSizeOfEngine.java
PRE-CREATION
ambari-server/src/test/java/org/apache/ambari/server/controller/metrics/timeline/MetricsPaddingMethodTest.java
c30c5eb
ambari-server/src/test/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineMetricCacheSizingTest.java
PRE-CREATION
ambari-server/src/test/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineMetricCacheTest.java
3432e1d
Diff: https://reviews.apache.org/r/39536/diff/
Testing
-------
Relevant unit test pass.
ambari-server:
AMSPropertyProviderTest
AMSReportPropertyProviderTest
StackDefinedPropertyProviderTest
TimelineMetricsCacheTest
TimleineMetricsCacheSizingTest
ambari-metrics:
All test passed
Thanks,
Sid Wagle
