-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44525/
-----------------------------------------------------------

Review request for Ambari and Sid Wagle.


Bugs: AMBARI-15331
    https://issues.apache.org/jira/browse/AMBARI-15331


Repository: ambari


Description
-------

AMBARI-15331: AMS HBase FIFO compaction policy and Normalizer settings are not 
handled correctly

A user could change the defaults for these settings via configuration updates 
but at least two problems would result.

1) Updates to variables AMS_HBASE_FIFO_COMPACTION_ENABLED and 
AMS_HBASE_NORMALIZER_ENABLED will not be honored. Same would be the case for 
the recent addition to ams-env - AMS_HBASE_INIT_CHECK_ENABLED.

These are hardcoded in ambari-metrics-collector rather than using what was 
propagated via ams-env.sh.To fix we'll try to use the values we expect to be 
determined by ams-env (which would also account for any user modification to 
the configuration). To be conservative we'll use some default value (same as it 
is today) but only if the variable was found not set.

2) During upgrade to Ambari 2.2.0 these settings will not be correctly handled 
due to incorrect variable names and leads to exceptions e.g if 
AMS_HBASE_FIFO_COMPACTION_ENABLED was expected to be set to 'false', for 
instance if the AMS HBase version did not support FIFO compaction policy.

HBASE_FIFO_COMPACTION_POLICY_ENABLED is used instead of 
AMS_HBASE_FIFO_COMPACTION_ENABLED
HBASE_NORMALIZATION_ENABLED is used instead of AMS_HBASE_NORMALIZER_ENABLED

Due to this and due to the hardcoded-values in ambari-metrics-collector 
undesired behavior (collector not starting up, attempts to startup AMS leaving 
hanging processes around) can result especially if the underlying AMS HBase 
version does not support FIFO compaction policy.


Diffs
-----

  
ambari-metrics/ambari-metrics-timelineservice/conf/unix/ambari-metrics-collector
 64a7848 
  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog220.java
 ac6b3c5 
  
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-env.xml
 78b8999 
  
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog220Test.java
 8263001 

Diff: https://reviews.apache.org/r/44525/diff/


Testing
-------

1) mvn clean test -fae -DambariVersion=2.2.0 -pl 
ambari-metrics/ambari-metrics-common,ambari-server

Tests run: 3929, Failures: 0, Errors: 2, Skipped: 33

Tests in error:
  PersistKeyValueImplTest.testMultiThreaded:103 » OutOfMemory unable to create 
n...
  AmbariManagementControllerTest.testCreateActionsFailures:4567 » Ambari Action 
...

(Errors unrelated to changes)


2) Updated Upgrade testcase for Jira.

3) Manually tested:

i) Set timeline.metrics.hbase.fifo.compaction.enabled to false and ensure it 
was effective after deployment. AMS components were all    started and running 
successfully with dashboards and heatmaps being displayed as expected.
ii) Upgrade from Ambari 2.1.0 to Ambari 2.2.0 and ensured the property 
timeline.metrics.hbase.fifo.compaction.enabled was false after deployment. AMS 
components were all started and running successfully with dashboards and 
heatmaps being displayed as expected.
iii) Updates to hbase.normalizer.enabled were effective in the configuration 
for both the new install and upgrade cases.


File Attachments
----------------

AMBARI-15331.patch
  
https://reviews.apache.org/media/uploaded/files/2016/03/08/243b5224-b255-404a-bd24-1b3ce5f79649__AMBARI-15331.patch


Thanks,

Shantanu Mundkur

Reply via email to