> On Oct. 22, 2015, 2:47 a.m., Sumit Mohanty wrote: > > ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineMetricsCacheSizeOfEngine.java, > > line 37 > > <https://reviews.apache.org/r/39536/diff/1/?file=1102879#file1102879line37> > > > > Probably a non-issue, but should it be 10000 as in initializeCache()
The SizeOfPolicyConfiguration overrides this, so real value is 10k. And yes, it doesn't really matter. - Sid ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/39536/#review103513 ----------------------------------------------------------- On Oct. 22, 2015, 1:46 a.m., Sid Wagle wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/39536/ > ----------------------------------------------------------- > > (Updated Oct. 22, 2015, 1:46 a.m.) > > > Review request for Ambari, Alejandro Fernandez, Dmytro Sen, Jonathan Hurley, > and Sumit Mohanty. > > > Bugs: AMBARI-13517 > https://issues.apache.org/jira/browse/AMBARI-13517 > > > Repository: ambari > > > Description > ------- > > - Max Heap restriction problem: Ehcache library that we use has a limit of > how far in the object graph it traverses to find size of an Cached reference, > this is to make it performant. Default limit = 1000. > - User gets following Warning which are harmless unless the data cannot fit > in memory and eviction does not kick in: > > WARN [qtp-client-70] ObjectGraphWalker:209 - The configured limit of 1,000 > object references was reached while attempting to calculate the size of the > object graph. Severe performance degradation could occur > if the sizing operation continues. This can be avoided by setting the > CacheManger or Cache <sizeOfPolicy> elements maxDepthExceededBehavior to > "abort" or adding stop points with @IgnoreSizeOf annotations. If performance > degradation > is NOT an issue at the configured limit, raise the limit value using the > CacheManager or Cache <sizeOfPolicy> elements maxDepth attribute. For more > information, see the Ehcache configuration documentation. > > _Objective of the patch_: > - Provide a custom Sizing engine for Ehcache that will provide a close > approximation of the data in the cache with significant perf gain. Premise: > Since we know the data structures we can make better estimates. > - Expectation from sizing engine : Discrepancy of less than 10K for a DS of > size 10 MB (Proven with a unit test) > > > Diffs > ----- > > > ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/TimelineMetric.java > 8b8df06 > > ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/cache/TimelineMetricsCache.java > 224b490 > > ambari-metrics/ambari-metrics-common/src/test/java/org/apache/hadoop/metrics2/sink/timeline/cache/TimelineMetricsCacheTest.java > 4a13d63 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java > 9c0b94d > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java > 9488316 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricReadHelper.java > 573e09d > > ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/ITClusterAggregator.java > b7b1737 > > ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/ITMetricAggregator.java > a3640d0 > > ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/MetricTestHelper.java > 26771d7 > > ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/MetricsPaddingMethod.java > 522c0bd > > ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineMetricCacheProvider.java > 6d80687 > > ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineMetricsCacheSizeOfEngine.java > PRE-CREATION > > ambari-server/src/test/java/org/apache/ambari/server/controller/metrics/timeline/MetricsPaddingMethodTest.java > c30c5eb > > ambari-server/src/test/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineMetricCacheSizingTest.java > PRE-CREATION > > ambari-server/src/test/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineMetricCacheTest.java > 3432e1d > > Diff: https://reviews.apache.org/r/39536/diff/ > > > Testing > ------- > > Relevant unit test pass. > ambari-server: > AMSPropertyProviderTest > AMSReportPropertyProviderTest > StackDefinedPropertyProviderTest > TimelineMetricsCacheTest > TimleineMetricsCacheSizingTest > ambari-metrics: > All test passed > > > Thanks, > > Sid Wagle > >
