Re: Review Request 47879: AMBARI-16807: After enabling HTTPS for HDFS, Data Node JVM Metrics on HDFS Heatmaps show NA
> On May 26, 2016, 3:11 p.m., Dmytro Sen wrote: > > Ship It! > > Qin Liu wrote: > Thank you so much Dmytro! > > Qin Liu wrote: > Hi Sid, > Could you please review the patch at your earliest convenience? Thank you > in advance! Hi Dmytro, I guess Sid is on vacation. I would appreciate if you can help push the patch to trunk. :-) - Qin --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47879/#review134989 --- On May 26, 2016, 10:27 a.m., Qin Liu wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/47879/ > --- > > (Updated May 26, 2016, 10:27 a.m.) > > > Review request for Ambari, Dmytro Sen and Sid Wagle. > > > Bugs: AMBARI-16807 > https://issues.apache.org/jira/browse/AMBARI-16807 > > > Repository: ambari > > > Description > --- > > After enabling HTTPS for HDFS, "DataNode Garbage Collection Time", "DataNode > JVM Heap Memory Used ", and "DataNode JVM Heap Memory Committed" widgets on > HDFS Heatmaps show NA. The root cause is that dfs.datanode.http.address > rather than dfs.datanode.https.address is used to get JMX metrics. > > Steps to reproduce: > 1. install a cluster with default from Ambari Web UI. > 2. configuring SSL for HDFS, YARN, and MapReduce. > 3. enable HTTPS for HDFS > Set the following properties in Advanced hdfs-site from Ambari Web UI: > dfs.http.policy=HTTPS_ONLY > dfs.datanode.http.address=0.0.0.0:50075 > dfs.datanode.https.address=0.0.0.0:50475 > 4. "DataNode Garbage Collection Time", "DataNode JVM Heap Memory Used " and > "DataNode JVM Heap Memory Committed" widgets on HDFS Heatmaps will show NA. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractProviderModule.java > 30ce19a > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/JMXHostProviderTest.java > 8ef6f0c > > Diff: https://reviews.apache.org/r/47879/diff/ > > > Testing > --- > > 1. The fix has been tested with HDP UI > "DataNode Garbage Collection Time", "DataNode JVM Heap Memory Used " and > "DataNode JVM Heap Memory Committed" metrics show up after apply the fix. > > 2. A new testcase is added in JMXHostProviderTest.java to verify JMX protocol > and port for DataNode. > Test set: org.apache.ambari.server.controller.internal.JMXHostProviderTest > --- > Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 111.192 sec > - in org.apache.ambari.server.controller.internal.JMXHostProviderTest > > 3. The ambari-server tests: > The following tests had issues in Hadoop QA testing but they passed in my > testing: > Failed tests: > > ConfigHelperTest$RunWithInMemoryDefaultTestModule.testCalculateIsStaleConfigsParallel:880 > null > Tests in error: > HostResourceProviderTest.testUpdateResourcesAsServiceAdministrator » > Unexpect... > > Here are my testing results for above tests: > Test set: org.apache.ambari.server.state.ConfigHelperTest > --- > Tests run: 18, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 160.88 sec - > in org.apache.ambari.server.state.ConfigHelperTest > > Test set: > org.apache.ambari.server.controller.internal.HostResourceProviderTest > --- > Tests run: 31, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.209 sec - > in org.apache.ambari.server.controller.internal.HostResourceProviderTest > > > Thanks, > > Qin Liu > >
Re: Review Request 47879: AMBARI-16807: After enabling HTTPS for HDFS, Data Node JVM Metrics on HDFS Heatmaps show NA
> On May 26, 2016, 3:11 p.m., Dmytro Sen wrote: > > Ship It! > > Qin Liu wrote: > Thank you so much Dmytro! Hi Sid, Could you please review the patch at your earliest convenience? Thank you in advance! - Qin --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47879/#review134989 --- On May 26, 2016, 10:27 a.m., Qin Liu wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/47879/ > --- > > (Updated May 26, 2016, 10:27 a.m.) > > > Review request for Ambari, Dmytro Sen and Sid Wagle. > > > Bugs: AMBARI-16807 > https://issues.apache.org/jira/browse/AMBARI-16807 > > > Repository: ambari > > > Description > --- > > After enabling HTTPS for HDFS, "DataNode Garbage Collection Time", "DataNode > JVM Heap Memory Used ", and "DataNode JVM Heap Memory Committed" widgets on > HDFS Heatmaps show NA. The root cause is that dfs.datanode.http.address > rather than dfs.datanode.https.address is used to get JMX metrics. > > Steps to reproduce: > 1. install a cluster with default from Ambari Web UI. > 2. configuring SSL for HDFS, YARN, and MapReduce. > 3. enable HTTPS for HDFS > Set the following properties in Advanced hdfs-site from Ambari Web UI: > dfs.http.policy=HTTPS_ONLY > dfs.datanode.http.address=0.0.0.0:50075 > dfs.datanode.https.address=0.0.0.0:50475 > 4. "DataNode Garbage Collection Time", "DataNode JVM Heap Memory Used " and > "DataNode JVM Heap Memory Committed" widgets on HDFS Heatmaps will show NA. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractProviderModule.java > 30ce19a > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/JMXHostProviderTest.java > 8ef6f0c > > Diff: https://reviews.apache.org/r/47879/diff/ > > > Testing > --- > > 1. The fix has been tested with HDP UI > "DataNode Garbage Collection Time", "DataNode JVM Heap Memory Used " and > "DataNode JVM Heap Memory Committed" metrics show up after apply the fix. > > 2. A new testcase is added in JMXHostProviderTest.java to verify JMX protocol > and port for DataNode. > Test set: org.apache.ambari.server.controller.internal.JMXHostProviderTest > --- > Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 111.192 sec > - in org.apache.ambari.server.controller.internal.JMXHostProviderTest > > 3. The ambari-server tests: > The following tests had issues in Hadoop QA testing but they passed in my > testing: > Failed tests: > > ConfigHelperTest$RunWithInMemoryDefaultTestModule.testCalculateIsStaleConfigsParallel:880 > null > Tests in error: > HostResourceProviderTest.testUpdateResourcesAsServiceAdministrator » > Unexpect... > > Here are my testing results for above tests: > Test set: org.apache.ambari.server.state.ConfigHelperTest > --- > Tests run: 18, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 160.88 sec - > in org.apache.ambari.server.state.ConfigHelperTest > > Test set: > org.apache.ambari.server.controller.internal.HostResourceProviderTest > --- > Tests run: 31, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.209 sec - > in org.apache.ambari.server.controller.internal.HostResourceProviderTest > > > Thanks, > > Qin Liu > >
Re: Review Request 47879: AMBARI-16807: After enabling HTTPS for HDFS, Data Node JVM Metrics on HDFS Heatmaps show NA
> On May 26, 2016, 3:11 p.m., Dmytro Sen wrote: > > Ship It! Thank you so much Dmytro! - Qin --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47879/#review134989 --- On May 26, 2016, 10:27 a.m., Qin Liu wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/47879/ > --- > > (Updated May 26, 2016, 10:27 a.m.) > > > Review request for Ambari, Dmytro Sen and Sid Wagle. > > > Bugs: AMBARI-16807 > https://issues.apache.org/jira/browse/AMBARI-16807 > > > Repository: ambari > > > Description > --- > > After enabling HTTPS for HDFS, "DataNode Garbage Collection Time", "DataNode > JVM Heap Memory Used ", and "DataNode JVM Heap Memory Committed" widgets on > HDFS Heatmaps show NA. The root cause is that dfs.datanode.http.address > rather than dfs.datanode.https.address is used to get JMX metrics. > > Steps to reproduce: > 1. install a cluster with default from Ambari Web UI. > 2. configuring SSL for HDFS, YARN, and MapReduce. > 3. enable HTTPS for HDFS > Set the following properties in Advanced hdfs-site from Ambari Web UI: > dfs.http.policy=HTTPS_ONLY > dfs.datanode.http.address=0.0.0.0:50075 > dfs.datanode.https.address=0.0.0.0:50475 > 4. "DataNode Garbage Collection Time", "DataNode JVM Heap Memory Used " and > "DataNode JVM Heap Memory Committed" widgets on HDFS Heatmaps will show NA. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractProviderModule.java > 30ce19a > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/JMXHostProviderTest.java > 8ef6f0c > > Diff: https://reviews.apache.org/r/47879/diff/ > > > Testing > --- > > 1. The fix has been tested with HDP UI > "DataNode Garbage Collection Time", "DataNode JVM Heap Memory Used " and > "DataNode JVM Heap Memory Committed" metrics show up after apply the fix. > > 2. A new testcase is added in JMXHostProviderTest.java to verify JMX protocol > and port for DataNode. > Test set: org.apache.ambari.server.controller.internal.JMXHostProviderTest > --- > Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 111.192 sec > - in org.apache.ambari.server.controller.internal.JMXHostProviderTest > > 3. The ambari-server tests: > The following tests had issues in Hadoop QA testing but they passed in my > testing: > Failed tests: > > ConfigHelperTest$RunWithInMemoryDefaultTestModule.testCalculateIsStaleConfigsParallel:880 > null > Tests in error: > HostResourceProviderTest.testUpdateResourcesAsServiceAdministrator » > Unexpect... > > Here are my testing results for above tests: > Test set: org.apache.ambari.server.state.ConfigHelperTest > --- > Tests run: 18, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 160.88 sec - > in org.apache.ambari.server.state.ConfigHelperTest > > Test set: > org.apache.ambari.server.controller.internal.HostResourceProviderTest > --- > Tests run: 31, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.209 sec - > in org.apache.ambari.server.controller.internal.HostResourceProviderTest > > > Thanks, > > Qin Liu > >
Re: Review Request 47879: AMBARI-16807: After enabling HTTPS for HDFS, Data Node JVM Metrics on HDFS Heatmaps show NA
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47879/#review134989 --- Ship it! Ship It! - Dmytro Sen On Май 26, 2016, 10:27 д.п., Qin Liu wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/47879/ > --- > > (Updated Май 26, 2016, 10:27 д.п.) > > > Review request for Ambari, Dmytro Sen and Sid Wagle. > > > Bugs: AMBARI-16807 > https://issues.apache.org/jira/browse/AMBARI-16807 > > > Repository: ambari > > > Description > --- > > After enabling HTTPS for HDFS, "DataNode Garbage Collection Time", "DataNode > JVM Heap Memory Used ", and "DataNode JVM Heap Memory Committed" widgets on > HDFS Heatmaps show NA. The root cause is that dfs.datanode.http.address > rather than dfs.datanode.https.address is used to get JMX metrics. > > Steps to reproduce: > 1. install a cluster with default from Ambari Web UI. > 2. configuring SSL for HDFS, YARN, and MapReduce. > 3. enable HTTPS for HDFS > Set the following properties in Advanced hdfs-site from Ambari Web UI: > dfs.http.policy=HTTPS_ONLY > dfs.datanode.http.address=0.0.0.0:50075 > dfs.datanode.https.address=0.0.0.0:50475 > 4. "DataNode Garbage Collection Time", "DataNode JVM Heap Memory Used " and > "DataNode JVM Heap Memory Committed" widgets on HDFS Heatmaps will show NA. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractProviderModule.java > 30ce19a > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/JMXHostProviderTest.java > 8ef6f0c > > Diff: https://reviews.apache.org/r/47879/diff/ > > > Testing > --- > > 1. The fix has been tested with HDP UI > "DataNode Garbage Collection Time", "DataNode JVM Heap Memory Used " and > "DataNode JVM Heap Memory Committed" metrics show up after apply the fix. > > 2. A new testcase is added in JMXHostProviderTest.java to verify JMX protocol > and port for DataNode. > Test set: org.apache.ambari.server.controller.internal.JMXHostProviderTest > --- > Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 111.192 sec > - in org.apache.ambari.server.controller.internal.JMXHostProviderTest > > 3. The ambari-server tests: > The following tests had issues in Hadoop QA testing but they passed in my > testing: > Failed tests: > > ConfigHelperTest$RunWithInMemoryDefaultTestModule.testCalculateIsStaleConfigsParallel:880 > null > Tests in error: > HostResourceProviderTest.testUpdateResourcesAsServiceAdministrator » > Unexpect... > > Here are my testing results for above tests: > Test set: org.apache.ambari.server.state.ConfigHelperTest > --- > Tests run: 18, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 160.88 sec - > in org.apache.ambari.server.state.ConfigHelperTest > > Test set: > org.apache.ambari.server.controller.internal.HostResourceProviderTest > --- > Tests run: 31, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.209 sec - > in org.apache.ambari.server.controller.internal.HostResourceProviderTest > > > Thanks, > > Qin Liu > >
Review Request 47879: AMBARI-16807: After enabling HTTPS for HDFS, Data Node JVM Metrics on HDFS Heatmaps show NA
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47879/ --- Review request for Ambari, Dmytro Sen and Sid Wagle. Bugs: AMBARI-16807 https://issues.apache.org/jira/browse/AMBARI-16807 Repository: ambari Description --- After enabling HTTPS for HDFS, "DataNode Garbage Collection Time", "DataNode JVM Heap Memory Used ", and "DataNode JVM Heap Memory Committed" widgets on HDFS Heatmaps show NA. The root cause is that dfs.datanode.http.address rather than dfs.datanode.https.address is used to get JMX metrics. Steps to reproduce: 1. install a cluster with default from Ambari Web UI. 2. configuring SSL for HDFS, YARN, and MapReduce. 3. enable HTTPS for HDFS Set the following properties in Advanced hdfs-site from Ambari Web UI: dfs.http.policy=HTTPS_ONLY dfs.datanode.http.address=0.0.0.0:50075 dfs.datanode.https.address=0.0.0.0:50475 4. "DataNode Garbage Collection Time", "DataNode JVM Heap Memory Used " and "DataNode JVM Heap Memory Committed" widgets on HDFS Heatmaps will show NA. Diffs - ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractProviderModule.java 30ce19a ambari-server/src/test/java/org/apache/ambari/server/controller/internal/JMXHostProviderTest.java 8ef6f0c Diff: https://reviews.apache.org/r/47879/diff/ Testing --- 1. The fix has been tested with HDP UI "DataNode Garbage Collection Time", "DataNode JVM Heap Memory Used " and "DataNode JVM Heap Memory Committed" metrics show up after apply the fix. 2. A new testcase is added in JMXHostProviderTest.java to verify JMX protocol and port for DataNode. Test set: org.apache.ambari.server.controller.internal.JMXHostProviderTest --- Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 111.192 sec - in org.apache.ambari.server.controller.internal.JMXHostProviderTest 3. The ambari-server tests: The following tests had issues in Hadoop QA testing but they passed in my testing: Failed tests: ConfigHelperTest$RunWithInMemoryDefaultTestModule.testCalculateIsStaleConfigsParallel:880 null Tests in error: HostResourceProviderTest.testUpdateResourcesAsServiceAdministrator » Unexpect... Here are my testing results for above tests: Test set: org.apache.ambari.server.state.ConfigHelperTest --- Tests run: 18, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 160.88 sec - in org.apache.ambari.server.state.ConfigHelperTest Test set: org.apache.ambari.server.controller.internal.HostResourceProviderTest --- Tests run: 31, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.209 sec - in org.apache.ambari.server.controller.internal.HostResourceProviderTest Thanks, Qin Liu
