Re: Review Request 44525: AMBARI-15331: AMS HBase FIFO compaction policy and Normalizer settings are not handled correctly
> On March 9, 2016, 8:13 p.m., Aravindan Vijayan wrote:
> > I think Point 1 will not be needed since we execute the ams-env.sh in
> > ambari-metrics-collector.
> >
> > # execute ams-env.sh
> > if [[ -f "${COLLECTOR_CONF_DIR}/ams-env.sh" ]]; then
> > . "${COLLECTOR_CONF_DIR}/ams-env.sh"
> > else
> > echo "ERROR: Cannot execute ${COLLECTOR_CONF_DIR}/ams-env.sh." 2>&1
> > exit 1
> > fi
> >
> >
> > Point 2 is a bug in UpgradeCatalog and the changes look fine.
> >
> > In any case, after https://issues.apache.org/jira/browse/AMBARI-15170,
> > these properties are not used in the ambari-metrics-collector startup
> > script.
>
> Shantanu Mundkur wrote:
> Aravindan,
>
> That is right. We don't really need any setting of the variables as we
> expect ams-env.sh to be sourced. I had half a mind to just remove the
> hardcoded lines for these but then that would be assuming that ams-env.sh
> would definitely set the necessary variables. So besides setting the
> variables after ams-env as been executed as an extra pre-caution chose to set
> them to what used to be hardcoded before. This is just in case and these
> statements are anyway no-ops if the variables were already set.
>
> BTW when we set FIFO compaction policy to True we have to be certain that
> the bundled HBase version supports it. Also, I thought the support for that
> was still experimental and so thought a default of False might have been
> preferable but Ambari chooses it as True out of the box. Thoughts?
>
> Thanks.
>
> Aravindan Vijayan wrote:
> Shantanu,
>
> I am fine with the changes to the startup script as well. As I mentioned
> though, these will not be used.
>
> We control what version of HBase is bundled with AMS in every Ambari
> version. That way we make sure the compatiblity between the HBase version and
> FIFO compaction policy. AMS HBase is not part of the stack that Ambari is
> managing on a cluster.
>
> Regarding FIFO compaction policy, we decided to go with a TRUE default
> value after consulting with a few HBase devs. We are still refining when and
> how we invoke these HBase config changes.
FIFO compaction gives us significant perf improvement, the normalizer is more
of a question mark from AMS point of view since it does merge Region boundaries.
We recently did test out both of these features on a 900+ node test cluster and
got positive results with only a single node AMS. (HA work is still on the
table)
- Sid
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44525/#review122790
---
On March 9, 2016, 8:59 p.m., Shantanu Mundkur wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44525/
> ---
>
> (Updated March 9, 2016, 8:59 p.m.)
>
>
> Review request for Ambari, Aravindan Vijayan and Sid Wagle.
>
>
> Bugs: AMBARI-15331
> https://issues.apache.org/jira/browse/AMBARI-15331
>
>
> Repository: ambari
>
>
> Description
> ---
>
> AMBARI-15331: AMS HBase FIFO compaction policy and Normalizer settings are
> not handled correctly
>
> A user could change the defaults for these settings via configuration updates
> but at least two problems would result.
>
> 1) Updates to variables AMS_HBASE_FIFO_COMPACTION_ENABLED and
> AMS_HBASE_NORMALIZER_ENABLED will not be honored. Same would be the case for
> the recent addition to ams-env - AMS_HBASE_INIT_CHECK_ENABLED.
>
> These are hardcoded in ambari-metrics-collector rather than using what was
> propagated via ams-env.sh.To fix we'll try to use the values we expect to be
> determined by ams-env (which would also account for any user modification to
> the configuration). To be conservative we'll use some default value (same as
> it is today) but only if the variable was found not set.
>
> 2) During upgrade to Ambari 2.2.0 these settings will not be correctly
> handled due to incorrect variable names and leads to exceptions e.g if
> AMS_HBASE_FIFO_COMPACTION_ENABLED was expected to be set to 'false', for
> instance if the AMS HBase version did not support FIFO compaction policy.
>
> HBASE_FIFO_COMPACTION_POLICY_ENABLED is used instead of
> AMS_HBASE_FIFO_COMPACTION_ENABLED
> HBASE_NORMALIZATION_ENABLED is used instead of AMS_HBASE_NORMALIZER_ENABLE
Review Request 44594: Fix ordering of Alter table calls which could result in Region Close issue
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44594/
---
Review request for Ambari, Aravindan Vijayan, Dmytro Sen, and Sumit Mohanty.
Bugs: AMBARI-15361
https://issues.apache.org/jira/browse/AMBARI-15361
Repository: ambari
Description
---
Two Alter table calls one of TTL and the other for setting the Region policies
could result in a possible race condition on HBase side.
Here is some details:
The assignment operation on the region in question
({{2e13a2d264c50e27c32cd4d3cad6ebca}}) further indicates possible network
partition - the region is CLOSED and OFFLINED successfully, and trying to OPEN
again, but it found conflict information from zk node (the zk node had a stale
information and still thought that it is still in the CLOSING state, which is
not true based on the log) - the assignment succeeds, but updating the
assignment information to zk failed due to incorrect version in the zk node.
Diffs
-
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java
f460292
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java
4afb722
Diff: https://reviews.apache.org/r/44594/diff/
Testing
---
Unit tests passed. No funtional change.
Thanks,
Sid Wagle
Re: Review Request 44594: Fix ordering of Alter table calls which could result in Region Close issue
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44594/
---
(Updated March 10, 2016, 12:17 a.m.)
Review request for Ambari, Aravindan Vijayan, Dmytro Sen, and Sumit Mohanty.
Changes
---
Modifications:
- Moved ALTER TTL calls to the HBaseAdmin call for updating policies
- Modified unit test to verify changes.
Bugs: AMBARI-15361
https://issues.apache.org/jira/browse/AMBARI-15361
Repository: ambari
Description
---
Two Alter table calls one of TTL and the other for setting the Region policies
could result in a possible race condition on HBase side.
Here is some details:
The assignment operation on the region in question
({{2e13a2d264c50e27c32cd4d3cad6ebca}}) further indicates possible network
partition - the region is CLOSED and OFFLINED successfully, and trying to OPEN
again, but it found conflict information from zk node (the zk node had a stale
information and still thought that it is still in the CLOSING state, which is
not true based on the log) - the assignment succeeds, but updating the
assignment information to zk failed due to incorrect version in the zk node.
Diffs (updated)
-
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java
465fe95
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java
48be4ee
ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/ITPhoenixHBaseAccessor.java
e3e037a
Diff: https://reviews.apache.org/r/44594/diff/
Testing
---
Unit tests passed. No funtional change.
Thanks,
Sid Wagle
Re: Review Request 44597: Postgres And c3p0 Queries Can Hang Ambari On Large Queries
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/44597/#review122845 --- Ship it! Ship It! - Sid Wagle On March 9, 2016, 10:30 p.m., Jonathan Hurley wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/44597/ > --- > > (Updated March 9, 2016, 10:30 p.m.) > > > Review request for Ambari, Alejandro Fernandez, Robert Levas, and Sid Wagle. > > > Bugs: AMBARI-15363 > https://issues.apache.org/jira/browse/AMBARI-15363 > > > Repository: ambari > > > Description > --- > > When executing certain JPA queries, the Ambari Server seems to deadlock in > the c3p0 library. It's caused by two issues: > > - Problems with c3p0 connection management (updated version to use better > connection pool handling) > - EclipseLink JPA CriteriaBuilder cartesian products (updated how we build > sorts for criteria to prevent duplicate Root) > > ``` > com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector@7675360a > -- APPARENT DEADLOCK!!! Creating emergency threads for unassigned pending > tasks! > 07 Mar 2016 18:11:01,108 WARN [Timer-0] ThreadPoolAsynchronousRunner:624 - > com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector@7675360a > -- APPARENT DEADLOCK!!! Complete Status: > Managed Threads: 3 > Active Threads: 1 > Active Tasks: > ``` > > Looks like the problem is twofold. First, we have a thread performing a very > heavy operation in the database: > > ``` > "qtp-ambari-client-36" prio=5 tid=0x7fd6b344f000 nid=0x8203 runnable > [0x72b3e000] >java.lang.Thread.State: RUNNABLE > at java.lang.Thread.currentThread(Native Method) > at java.lang.ThreadLocal.get(ThreadLocal.java:143) > at java.lang.StringCoding.deref(StringCoding.java:63) > at java.lang.StringCoding.decode(StringCoding.java:179) > at java.lang.String.(String.java:416) > at org.postgresql.core.Encoding.decode(Encoding.java:191) > at org.postgresql.core.Encoding.decode(Encoding.java:203) > at > org.postgresql.jdbc2.AbstractJdbc2ResultSet.getString(AbstractJdbc2ResultSet.java:1979) > at > com.mchange.v2.c3p0.impl.NewProxyResultSet.getString(NewProxyResultSet.java:3316) > ... > at > org.apache.ambari.server.orm.dao.DaoUtils.selectList(DaoUtils.java:62) > ``` > > And then we see this: > > ``` > Internal Exception: org.postgresql.util.PSQLException: Ran out of memory > retrieving query results. > Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded > Mar 08, 2016 9:57:53 PM com.sun.jersey.spi.container.ContainerResponse > mapMappableContainerException > SEVERE: The exception contained within MappableContainerException could not > be mapped to a response, re-throwing to the HTTP container > java.lang.OutOfMemoryError: GC overhead limit exceeded > at java.lang.StringCoding$StringDecoder.decode(StringCoding.java:149) > at java.lang.StringCoding.decode(StringCoding.java:193) > at java.lang.String.(String.java:416) > at org.postgresql.core.Encoding.decode(Encoding.java:191) > at org.postgresql.core.Encoding.decode(Encoding.java:203) > at > org.postgresql.jdbc2.AbstractJdbc2ResultSet.getString(AbstractJdbc2ResultSet.java:1979) > at > com.mchange.v2.c3p0.impl.NewProxyResultSet.getString(NewProxyResultSet.java:3316) > ... > at > org.eclipse.persistence.internal.jpa.QueryImpl.executeReadQuery(QueryImpl.java:258) > ``` > > > Diffs > - > > ambari-project/pom.xml ed94004 > ambari-server/pom.xml f691fad > > ambari-server/src/main/java/org/apache/ambari/server/api/query/JpaPredicateVisitor.java > 3a8a631 > > ambari-server/src/main/java/org/apache/ambari/server/api/query/JpaSortBuilder.java > 8021346 > > Diff: https://reviews.apache.org/r/44597/diff/ > > > Testing > --- > > Executed problematic queries against a massive database. Problem is resolved > in test environment. > > Unit Tests Pending > > > Thanks, > > Jonathan Hurley > >
Re: Review Request 44594: Fix ordering of Alter table calls which could result in Region Close issue
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44594/
---
(Updated March 10, 2016, 12:34 a.m.)
Review request for Ambari, Aravindan Vijayan, Dmytro Sen, and Sumit Mohanty.
Changes
---
Changed.
Bugs: AMBARI-15361
https://issues.apache.org/jira/browse/AMBARI-15361
Repository: ambari
Description
---
Two Alter table calls one of TTL and the other for setting the Region policies
could result in a possible race condition on HBase side.
Here is some details:
The assignment operation on the region in question
({{2e13a2d264c50e27c32cd4d3cad6ebca}}) further indicates possible network
partition - the region is CLOSED and OFFLINED successfully, and trying to OPEN
again, but it found conflict information from zk node (the zk node had a stale
information and still thought that it is still in the CLOSING state, which is
not true based on the log) - the assignment succeeds, but updating the
assignment information to zk failed due to incorrect version in the zk node.
Diffs (updated)
-
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java
465fe95
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java
48be4ee
ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/ITPhoenixHBaseAccessor.java
e3e037a
Diff: https://reviews.apache.org/r/44594/diff/
Testing
---
Unit tests passed. No funtional change.
Thanks,
Sid Wagle
Re: Review Request 44525: AMBARI-15331: AMS HBase FIFO compaction policy and Normalizer settings are not handled correctly
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/44525/#review122872 --- Ship it! Ship It! - Sid Wagle On March 9, 2016, 8:59 p.m., Shantanu Mundkur wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/44525/ > --- > > (Updated March 9, 2016, 8:59 p.m.) > > > Review request for Ambari, Aravindan Vijayan and Sid Wagle. > > > Bugs: AMBARI-15331 > https://issues.apache.org/jira/browse/AMBARI-15331 > > > Repository: ambari > > > Description > --- > > AMBARI-15331: AMS HBase FIFO compaction policy and Normalizer settings are > not handled correctly > > A user could change the defaults for these settings via configuration updates > but at least two problems would result. > > 1) Updates to variables AMS_HBASE_FIFO_COMPACTION_ENABLED and > AMS_HBASE_NORMALIZER_ENABLED will not be honored. Same would be the case for > the recent addition to ams-env - AMS_HBASE_INIT_CHECK_ENABLED. > > These are hardcoded in ambari-metrics-collector rather than using what was > propagated via ams-env.sh.To fix we'll try to use the values we expect to be > determined by ams-env (which would also account for any user modification to > the configuration). To be conservative we'll use some default value (same as > it is today) but only if the variable was found not set. > > 2) During upgrade to Ambari 2.2.0 these settings will not be correctly > handled due to incorrect variable names and leads to exceptions e.g if > AMS_HBASE_FIFO_COMPACTION_ENABLED was expected to be set to 'false', for > instance if the AMS HBase version did not support FIFO compaction policy. > > HBASE_FIFO_COMPACTION_POLICY_ENABLED is used instead of > AMS_HBASE_FIFO_COMPACTION_ENABLED > HBASE_NORMALIZATION_ENABLED is used instead of AMS_HBASE_NORMALIZER_ENABLED > > Due to this and due to the hardcoded-values in ambari-metrics-collector > undesired behavior (collector not starting up, attempts to startup AMS > leaving hanging processes around) can result especially if the underlying AMS > HBase version does not support FIFO compaction policy. > > > Diffs > - > > > ambari-metrics/ambari-metrics-timelineservice/conf/unix/ambari-metrics-collector > 64a7848 > > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog220.java > ac6b3c5 > > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-env.xml > 78b8999 > > ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog220Test.java > 8263001 > > Diff: https://reviews.apache.org/r/44525/diff/ > > > Testing > --- > > 1) Results : > > Tests run: 3930, Failures: 0, Errors: 0, Skipped: 33 > > 2) Updated UpgradeCatalog220Test testcase. > > 3) Manually tested: > > i) Set timeline.metrics.hbase.fifo.compaction.enabled to false and ensure it > was effective after deployment. AMS components were allstarted and > running successfully with dashboards and heatmaps being displayed as expected. > ii) Upgrade from Ambari 2.1.0 to Ambari 2.2.0 and ensured the property > timeline.metrics.hbase.fifo.compaction.enabled was false after deployment. > AMS components were all started and running successfully with dashboards and > heatmaps being displayed as expected. > iii) Updates to hbase.normalizer.enabled were effective in the configuration > for both the new install and upgrade cases. > > > File Attachments > > > AMBARI-15331.patch > > https://reviews.apache.org/media/uploaded/files/2016/03/08/243b5224-b255-404a-bd24-1b3ce5f79649__AMBARI-15331.patch > > > Thanks, > > Shantanu Mundkur > >
Re: Review Request 44811: AMBARI-15413 : Metrics Grafana Failed to start
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/44811/#review123545 --- Ship it! Ship It! - Sid Wagle On March 14, 2016, 11:47 p.m., Aravindan Vijayan wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/44811/ > --- > > (Updated March 14, 2016, 11:47 p.m.) > > > Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle. > > > Bugs: AMBARI-15413 > https://issues.apache.org/jira/browse/AMBARI-15413 > > > Repository: ambari > > > Description > --- > > Restart Grafana does not work after changing AMS user. > > > Diffs > - > > > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/ams.py > 63ae0a9 > > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_grafana.py > 76e8e20 > > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_grafana_util.py > 593591c > > ambari-server/src/test/python/stacks/2.0.6/AMBARI_METRICS/test_metrics_grafana.py > 43d5ad0 > > Diff: https://reviews.apache.org/r/44811/diff/ > > > Testing > --- > > Manual testing done. > > Python unit tests pass. > > > Thanks, > > Aravindan Vijayan > >
Review Request 44816: Fix new HDFS AMS alerts to account for NN rpc ports
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/44816/ --- Review request for Ambari, Aravindan Vijayan, Dmytro Sen, and Sumit Mohanty. Bugs: AMBARI-15415 https://issues.apache.org/jira/browse/AMBARI-15415 Repository: ambari Description --- AMBARI-15293, changed the metric names for rpc and rpcdetailed metrics for NN based on the port configuration for client / datanode / healthcheck rpc We need a fix this by cloning the alert definition that we have into 2 different alerts for Client / DN ports. *Note*: By default client and DN RPC ports are the same 8020 and for this case DN rpc alert should be a NOOP. Diffs - ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java 611cb92 ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/alerts.json 2a6229c ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/alerts/alert_metrics_deviation.py f6a9a56 Diff: https://reviews.apache.org/r/44816/diff/ Testing --- Manually verified, python unit tests passed. Thanks, Sid Wagle
Review Request 44817: Grafana password is required field after upgrade from 2.2.0 to 2.2.2
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/44817/ --- Review request for Ambari, Aravindan Vijayan, Dmytro Sen, and Jaimin Jetly. Bugs: AMBARI-15416 https://issues.apache.org/jira/browse/AMBARI-15416 Repository: ambari Description --- The metrics_grafana_password shows up as the required field even though Grafana component is not added. Diffs - ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 6c62f0f ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py 8d8938f Diff: https://reviews.apache.org/r/44817/diff/ Testing --- -- Ran 246 tests in 6.575s OK -- Total run:908 Total errors:0 Total failures:0 OK Thanks, Sid Wagle
Review Request 44941: Report AMS / Grafana Per Disk Metrics and other changes
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/44941/ --- Review request for Ambari, Aravindan Vijayan, Dmytro Sen, and Sumit Mohanty. Bugs: AMBARI-15448 https://issues.apache.org/jira/browse/AMBARI-15448 Repository: ambari Description --- We need to collect and expose per-disk metrics from AMS for iostats counters. We can sort the disk names alphabetically, and name the metrics like _1, _2, etc. Plus from psutils: busy_time: (Linux, FreeBSD) time spent doing actual I/Os (in milliseconds) read_merged_count (Linux): number of merged reads (see iostat doc) write_merged_count (Linux): number of merged writes (see iostats doc) _Additional fixes_: - Combined_disk_total metric seems to be wrong : it is a sum() of all disk percentages from different disks - Missing ATS and JHS metrics due to wrong config item in hadoop-metrics2.properties file - Report number of disks Diffs - ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/config_reader.py a053955 ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/controller.py c04a61b ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/host_info.py 2e8c442 ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/metric_collector.py c28fd03 ambari-metrics/ambari-metrics-host-monitoring/src/test/python/core/TestHostInfo.py f07b573 ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_grafana_util.py e5020a5 ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/templates/metric_monitor.ini.j2 0b0932a ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/templates/hadoop-metrics2.properties.j2 47b504f Diff: https://reviews.apache.org/r/44941/diff/ Testing --- Manually verified. -- Ran 12 tests in 0.020s OK -- Total run:12 Total errors:0 Total failures:0 OK Thanks, Sid Wagle
Re: Review Request 44941: Report AMS / Grafana Per Disk Metrics and other changes
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/44941/ --- (Updated March 17, 2016, 1:09 a.m.) Review request for Ambari, Aravindan Vijayan, Dmytro Sen, and Sumit Mohanty. Changes --- - Added swap_in and swap_out - Fixed memory used to reflect (used - cached) Bugs: AMBARI-15448 https://issues.apache.org/jira/browse/AMBARI-15448 Repository: ambari Description --- We need to collect and expose per-disk metrics from AMS for iostats counters. We can sort the disk names alphabetically, and name the metrics like _1, _2, etc. Plus from psutils: busy_time: (Linux, FreeBSD) time spent doing actual I/Os (in milliseconds) read_merged_count (Linux): number of merged reads (see iostat doc) write_merged_count (Linux): number of merged writes (see iostats doc) _Additional fixes_: - Combined_disk_total metric seems to be wrong : it is a sum() of all disk percentages from different disks - Missing ATS and JHS metrics due to wrong config item in hadoop-metrics2.properties file - Report number of disks Diffs (updated) - ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/config_reader.py a053955 ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/controller.py c04a61b ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/host_info.py 2e8c442 ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/metric_collector.py c28fd03 ambari-metrics/ambari-metrics-host-monitoring/src/test/python/core/TestHostInfo.py f07b573 ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_grafana_util.py e5020a5 ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/templates/metric_monitor.ini.j2 0b0932a ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/templates/hadoop-metrics2.properties.j2 47b504f Diff: https://reviews.apache.org/r/44941/diff/ Testing --- Manually verified. -- Ran 12 tests in 0.020s OK -- Total run:12 Total errors:0 Total failures:0 OK Thanks, Sid Wagle
Re: Review Request 44916: StackAdvisor needs to set config map for ams-grafana-env to meet non-null expectations
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/44916/#review123895 --- Shouldn't we also have a non-null check in the backend code? - Sid Wagle On March 16, 2016, 4:59 p.m., Robert Levas wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/44916/ > --- > > (Updated March 16, 2016, 4:59 p.m.) > > > Review request for Ambari, Dmytro Sen, Jonathan Hurley, Nate Cole, and Sid > Wagle. > > > Bugs: AMBARI-15442 > https://issues.apache.org/jira/browse/AMBARI-15442 > > > Repository: ambari > > > Description > --- > > The StackAdvisor needs to set a config map for the `ams-grafana-env`, even it > is empty, to meet non-null expectations in Kerberos and Blueprint logic. > > To to this, the following line needs to be added to the StackAdvisor code: > > #stacks/HDP/2.0.6/services/stack_advisor.py:473 > ``` > putGrafanaProperty = self.putProperty(configurations, "ams-grafana-env", > services) > ``` > > > Diffs > - > > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py > 8c18421 > > Diff: https://reviews.apache.org/r/44916/diff/ > > > Testing > --- > > Manually tested in cluster Kerbrerized using the Ambari UI. > > > Thanks, > > Robert Levas > >
Re: Review Request 44916: StackAdvisor needs to set config map for ams-grafana-env to meet non-null expectations
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/44916/#review123893 --- Ship it! Ship It! - Sid Wagle On March 16, 2016, 4:59 p.m., Robert Levas wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/44916/ > --- > > (Updated March 16, 2016, 4:59 p.m.) > > > Review request for Ambari, Dmytro Sen, Jonathan Hurley, Nate Cole, and Sid > Wagle. > > > Bugs: AMBARI-15442 > https://issues.apache.org/jira/browse/AMBARI-15442 > > > Repository: ambari > > > Description > --- > > The StackAdvisor needs to set a config map for the `ams-grafana-env`, even it > is empty, to meet non-null expectations in Kerberos and Blueprint logic. > > To to this, the following line needs to be added to the StackAdvisor code: > > #stacks/HDP/2.0.6/services/stack_advisor.py:473 > ``` > putGrafanaProperty = self.putProperty(configurations, "ams-grafana-env", > services) > ``` > > > Diffs > - > > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py > 8c18421 > > Diff: https://reviews.apache.org/r/44916/diff/ > > > Testing > --- > > Manually tested in cluster Kerbrerized using the Ambari UI. > > > Thanks, > > Robert Levas > >
Re: Review Request 45125: 'ambari-server check-database' has errors after upgrade to ambari 2.2.2.0 [grafana errors]
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45125/#review124647 --- Ship it! Ship It! - Sid Wagle On March 21, 2016, 8:12 p.m., Vitalyi Brodetskyi wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45125/ > --- > > (Updated March 21, 2016, 8:12 p.m.) > > > Review request for Ambari, Sumit Mohanty and Sid Wagle. > > > Bugs: AMBARI-15495 > https://issues.apache.org/jira/browse/AMBARI-15495 > > > Repository: ambari > > > Description > --- > > STR: > 1)Deploy old version > 2)Make Ambari only Upgrade > 4)'ambari-server check-database' > Actual result: > 'ambari-server check-database' has errors after upgrade to ambari 2.2.2.0 > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/checks/CheckDatabaseHelper.java > 652a49f > > ambari-server/src/test/java/org/apache/ambari/server/checks/CheckDatabaseHelperTest.java > 66c6667 > > Diff: https://reviews.apache.org/r/45125/diff/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Vitalyi Brodetskyi > >
Re: Review Request 45150: AMBARI-15476 : [AMS / Grafana] Rate calculation with sum() is wrong
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45150/#review124856 --- ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/PostProcessingUtil.java (line 32) <https://reviews.apache.org/r/45150/#comment187545> Why create a new TreeMap? ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricClusterAggregatorSecond.java (line 81) <https://reviews.apache.org/r/45150/#comment187548> Add comment why this is changed this way. ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricClusterAggregatorSecond.java (line 112) <https://reviews.apache.org/r/45150/#comment187549> Make this package private instead for visiblity to tests. ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricClusterAggregatorSecond.java (line 190) <https://reviews.apache.org/r/45150/#comment187550> Same ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricClusterAggregatorSecond.java (line 234) <https://reviews.apache.org/r/45150/#comment187553> Braces. ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricClusterAggregatorSecond.java (line 235) <https://reviews.apache.org/r/45150/#comment187552> formatting ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricClusterAggregatorSecond.java (line 259) <https://reviews.apache.org/r/45150/#comment187557> format ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog222.java (line 315) <https://reviews.apache.org/r/45150/#comment187558> I think 72 hours makes sense since it goes past a 2 day weekend. Add a config to disable interpolation all together. - Sid Wagle On March 22, 2016, 5:08 a.m., Aravindan Vijayan wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45150/ > --- > > (Updated March 22, 2016, 5:08 a.m.) > > > Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle. > > > Bugs: AMBARI-15476 > https://issues.apache.org/jira/browse/AMBARI-15476 > > > Repository: ambari > > > Description > --- > > Fix TimelineMetricAggregatorSecond such that time slices with no datapoints > for a series are given interpolated values based on nearby values. > > Change METRIC_RECORD TTL to 2 days in distributed mode. > > Add stack advisor recommendation for AMS HBase log directory based on AMS log > directory. > > > Diffs > - > > > ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/PostProcessingUtil.java > PRE-CREATION > > ambari-metrics/ambari-metrics-common/src/test/java/org/apache/hadoop/metrics2/sink/timeline/cache/PostProcessingUtilTest.java > PRE-CREATION > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricClusterAggregatorSecond.java > e8e16a7 > > ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricClusterAggregatorSecondTest.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog222.java > 6c8d153 > > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-hbase-env.xml > b99d1f9 > > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml > 7a8984a > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py > d33e081 > > ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog222Test.java > 599ac3e > ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py > 41da868 > > Diff: https://reviews.apache.org/r/45150/diff/ > > > Testing > --- > > Manual testing done. > > Added unit tests. > > > Thanks, > > Aravindan Vijayan > >
Re: Review Request 45150: AMBARI-15476 : [AMS / Grafana] Rate calculation with sum() is wrong
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45150/#review124866 --- Ship it! Ship It! - Sid Wagle On March 22, 2016, 7:44 p.m., Aravindan Vijayan wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45150/ > --- > > (Updated March 22, 2016, 7:44 p.m.) > > > Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle. > > > Bugs: AMBARI-15476 > https://issues.apache.org/jira/browse/AMBARI-15476 > > > Repository: ambari > > > Description > --- > > Fix TimelineMetricAggregatorSecond such that time slices with no datapoints > for a series are given interpolated values based on nearby values. > > Change METRIC_RECORD TTL to 2 days in distributed mode. > > Add stack advisor recommendation for AMS HBase log directory based on AMS log > directory. > > > Diffs > - > > > ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/PostProcessingUtil.java > PRE-CREATION > > ambari-metrics/ambari-metrics-common/src/test/java/org/apache/hadoop/metrics2/sink/timeline/cache/PostProcessingUtilTest.java > PRE-CREATION > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricConfiguration.java > aa12735 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricClusterAggregatorSecond.java > e8e16a7 > > ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricClusterAggregatorSecondTest.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog222.java > 6c8d153 > > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-hbase-env.xml > b99d1f9 > > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml > 7a8984a > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py > 61aa2ba > > ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog222Test.java > 599ac3e > ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py > 41da868 > > Diff: https://reviews.apache.org/r/45150/diff/ > > > Testing > --- > > Manual testing done. > > Added unit tests. > > > Thanks, > > Aravindan Vijayan > >
Re: Review Request 45215: HDFS Alerts for AMS Throw 'invalid literal for int() with base 10: '50.0''
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45215/#review125036 --- Ship it! Ship It! - Sid Wagle On March 23, 2016, 3:08 p.m., Jonathan Hurley wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45215/ > --- > > (Updated March 23, 2016, 3:08 p.m.) > > > Review request for Ambari, Nate Cole and Sid Wagle. > > > Bugs: AMBARI-15533 > https://issues.apache.org/jira/browse/AMBARI-15533 > > > Repository: ambari > > > Description > --- > > SCRIPT alerts stuck in UNKNWN status with response message 'invalid literal > for int() with base 10: '50.0''. > > It is noticed that the error is thrown only after a PUT alertDefinition call > to update few parameters of alert definition since the numeric values are > changed into strings. > > The scripts need to safely cast their parameters; the fix is in the script > here. > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/alerts/alert_metrics_deviation.py > f62c4a3 > > Diff: https://reviews.apache.org/r/45215/diff/ > > > Testing > --- > > Deployed on a cluster exibiting the cast problem. > > > Thanks, > > Jonathan Hurley > >
Re: Review Request 45250: AMBARI-15540 : NAMENODE critical alert is present [Percentage standard deviation] after upgrade from 2.0.2/ 2.2.1.0 etc to 2.2.2.0 and disabling security
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45250/#review125116
---
Ship it!
Ship It!
- Sid Wagle
On March 23, 2016, 8:26 p.m., Aravindan Vijayan wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45250/
> ---
>
> (Updated March 23, 2016, 8:26 p.m.)
>
>
> Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle.
>
>
> Bugs: AMBARI-15540
> https://issues.apache.org/jira/browse/AMBARI-15540
>
>
> Repository: ambari
>
>
> Description
> ---
>
> STR:
> 1)Deploy old version
> 2)Enable Mit security
> 3)Make Ambari only Upgrade
> 4)Disable security
> 5)Enable NN, RM HA
> 6)Enable/Disable security
> 7) Check Alerts for HDFS
>
> *Issue was absent for*
> {code}
> 2.2.2.0-285
> 2b1e7b75d1970f26b0a033cdfac8b53b457b83ff
> {code}
>
> *Actual result:*
> NAMENODE critical alert is present [Percentage standard deviation] after
> upgrade from 2.0.2/ 2.2.1.0 etc to 2.2.2.0 and disabling security
> {code}
> {
> "href" : "https://<>:8443/api/v1/clusters/cl1/alerts/201",
> "Alert" : {
> "cluster_name" : "cl1",
> "component_name" : "NAMENODE",
> "definition_id" : 102,
> "definition_name" : "namenode_client_rpc_processing_latency_daily",
> "host_name" : "<>",
> "id" : 201,
> "instance" : null,
> "label" : "NameNode Client RPC Processing Latency (Daily)",
> "latest_timestamp" : 1458643485805,
> "maintenance_state" : "OFF",
> "original_timestamp" : 1458643485805,
> "scope" : "ANY",
> "service_name" : "HDFS",
> "state" : "CRITICAL",
> "text" : "CRITICAL. Percentage standard deviation value 218.17% is
> beyond the critical threshold of 200.00%"
> }
> },
> {code}
>
>
> Diffs
> -
>
>
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/alerts/alert_metrics_deviation.py
> f62c4a3
>
> Diff: https://reviews.apache.org/r/45250/diff/
>
>
> Testing
> ---
>
>
> Thanks,
>
> Aravindan Vijayan
>
>
Re: Review Request 45395: Ambari API does not return HDFS RPC metrics based on ports
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45395/#review125784
---
Fix it, then Ship it!
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractProviderModule.java
(line 102)
<https://reviews.apache.org/r/45395/#comment188659>
Doesn't this need to be ConcurrentHashMap?
ambari-server/src/main/java/org/apache/ambari/server/controller/jmx/JMXHostProvider.java
(line 81)
<https://reviews.apache.org/r/45395/#comment188660>
Decription doesn't match function.
- Sid Wagle
On March 28, 2016, 6:19 p.m., Dmytro Sen wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45395/
> ---
>
> (Updated March 28, 2016, 6:19 p.m.)
>
>
> Review request for Ambari and Sid Wagle.
>
>
> Bugs: AMBARI-15606
> https://issues.apache.org/jira/browse/AMBARI-15606
>
>
> Repository: ambari
>
>
> Description
> ---
>
> ISSUE: Ambari API does not return HDFS RPC metrics based on ports (Example
> 8020 and 8040)
> When creating Ambari Widget we don't see an option to monitor by port. Hence
> the monitoring of Random port or aggregate doesn't give correct information
>
> After the patch, ambari automatically replaces port number with the
> corresponding suffix
> client for the port specified in hdfs-site/dfs.namenode.rpc-address
> datanode for the port specified in hdfs-site/dfs.namenode.servicerpc-address
> healthcheck for the port specified in
> hdfs-site/dfs.namenode.lifeline.rpc-address
>
> Request/response example
>
> {
> "href" :
> "http://10.2.2.22:1081/api/v1/clusters/c1/services/HDFS/components/NAMENODE?fields=host_components/metrics/rpc/*,host_components/metrics/runtime/*";,
> "ServiceComponentInfo" : {
> "cluster_name" : "c1",
> "component_name" : "NAMENODE",
> "service_name" : "HDFS"
> },
> "host_components" : [
> {
> "href" :
> "http://10.2.2.22:1081/api/v1/clusters/c1/hosts/c6401.ambari.apache.org/host_components/NAMENODE";,
> "HostRoles" : {
> "cluster_name" : "c1",
> "component_name" : "NAMENODE",
> "host_name" : "c6401.ambari.apache.org"
> },
> "metrics" : {
> "rpc" : {
> "client" : {
> "NumOpenConnections" : 0,
> "ReceivedBytes" : 2508,
> "RpcProcessingTime_avg_time" : 0.0,
> "RpcProcessingTime_num_ops" : 12,
> "RpcQueueTime_avg_time" : 0.0,
> "RpcQueueTime_num_ops" : 12,
> "SentBytes" : 408,
> "callQueueLen" : 0,
> "rpcAuthenticationFailures" : 0,
> "rpcAuthenticationSuccesses" : 0,
> "rpcAuthorizationFailures" : 0,
> "rpcAuthorizationSuccesses" : 12
> },
> "datanode" : {
> "NumOpenConnections" : 2,
> "ReceivedBytes" : 3269926,
> "RpcProcessingTime_avg_time" : 0.0,
> "RpcProcessingTime_num_ops" : 6029,
> "RpcQueueTime_avg_time" : 0.0,
> "RpcQueueTime_num_ops" : 6029,
> "SentBytes" : 248169,
> "callQueueLen" : 0,
> "rpcAuthenticationFailures" : 0,
> "rpcAuthenticationSuccesses" : 0,
> "rpcAuthorizationFailures" : 0,
> "rpcAuthorizationSuccesses" : 288
> }
> },
> "runtime" : {
> "StartTime" : 1459170322047
> }
> }
> }
> ]
> }
>
>
> Diffs
> -
>
>
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java
> 4b15b80
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractProviderModule.java
> 7747753
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/jmx/JMXHostProvider.java
> 15d44b2
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/jmx/JMXPropertyProvider.java
> b883d2b
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/utilities/PropertyHelper.java
> 41da279
>
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/params.py
> b0e2e7a
>
> ambari-server/src/test/java/org/apache/ambari/server/controller/metrics/JMXPropertyProviderTest.java
> 35ba605
>
> ambari-server/src/test/java/org/apache/ambari/server/controller/utilities/PropertyHelperTest.java
> 2beb462
>
> Diff: https://reviews.apache.org/r/45395/diff/
>
>
> Testing
> ---
>
> Unit tests passed
>
>
> Thanks,
>
> Dmytro Sen
>
>
Review Request 45465: Support distributed aggregation for multiple AMS instances
a 9c7c8fa ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricClusterAggregatorSecondTest.java d2d478c ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/availability/TimelineMetricHAControllerTest.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java 38a3614 ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-env.xml 836e159 ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/metainfo.xml b06f55b ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog240Test.java 7b2d797 Diff: https://reviews.apache.org/r/45465/diff/ Testing --- Manually verified. Thanks, Sid Wagle
Re: Review Request 45465: Support distributed aggregation for multiple AMS instances
> On March 30, 2016, 5:15 a.m., Sumit Mohanty wrote: > > ambari-metrics/ambari-metrics-timelineservice/pom.xml, line 268 > > <https://reviews.apache.org/r/45465/diff/1/?file=1318600#file1318600line268> > > > > Understand that ZK version hardly changes but should this be > > parameterized as well so that each vendor can pick their own version? Had to do this because of version discrepancy resulting in unit test failing. Verfied helix compat with this version. > On March 30, 2016, 5:15 a.m., Sumit Mohanty wrote: > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java, > > line 104 > > <https://reviews.apache.org/r/45465/diff/1/?file=1318601#file1318601line104> > > > > do we bail out here? Yes, since dist mode is unabled, the bail out here means there was an issue talking to ZK of the cluster. > On March 30, 2016, 5:15 a.m., Sumit Mohanty wrote: > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricStore.java, > > line 93 > > <https://reviews.apache.org/r/45465/diff/1/?file=1318603#file1318603line93> > > > > ports? could collector run on a custom port? Helix controller does not listen on any port, the API port is redundant since client getting here needs to know what it is. > On March 30, 2016, 5:15 a.m., Sumit Mohanty wrote: > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/AbstractTimelineAggregator.java, > > line 142 > > <https://reviews.apache.org/r/45465/diff/1/?file=1318604#file1318604line142> > > > > Lets add the aggregation type being skipped. This is covered because the Logger is initialized with the custom name of the aggregator. So everything that is printed has well defined tracebility of which thread it came from. I have done same thing for jstack so we can differentiate based on thread name. - Sid --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45465/#review126026 --- On March 30, 2016, 12:49 a.m., Sid Wagle wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45465/ > --- > > (Updated March 30, 2016, 12:49 a.m.) > > > Review request for Ambari, Aravindan Vijayan, Dmytro Sen, and Sumit Mohanty. > > > Bugs: AMBARI-15623 > https://issues.apache.org/jira/browse/AMBARI-15623 > > > Repository: ambari > > > Description > --- > > (Preliminary patch - verified funtinoally and added a unit test) > > Tasks: > > Support "distributed" mode in AMS, this implies following changes to the > collector service: > > - Collectors discover each other > - Add a service to discover all collector by knowing any one > - Aggregation functions are distributed among collectors automatically with > failover tolerance > - Save checkpoint data in Zookeeper > > > Diffs > - > > ambari-metrics/ambari-metrics-timelineservice/pom.xml b435964 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java > 2f080e3 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricConfiguration.java > e57f02d > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricStore.java > 0aa102e > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/AbstractTimelineAggregator.java > c5b9ba1 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricAggregator.java > 295db0e > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricAggregatorFactory.java > cc85c56 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline
Re: Review Request 45447: AMBARI-15621 : Cluster Second aggregator taking more than 2 mins to execute on large clusters, thereby causing lag.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45447/#review126140 --- Fix it, then Ship it! ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/AbstractTimelineAggregator.java (line 105) <https://reviews.apache.org/r/45447/#comment189037> Should be consistent time use for cal and logged. - Sid Wagle On March 29, 2016, 9:15 p.m., Aravindan Vijayan wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45447/ > --- > > (Updated March 29, 2016, 9:15 p.m.) > > > Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle. > > > Bugs: AMBARI-15621 > https://issues.apache.org/jira/browse/AMBARI-15621 > > > Repository: ambari > > > Description > --- > > Fix issues in aggregator scheduling implementation. > > Recommend value of "timeline.metrics.service.resultset.fetchSize" through > stack advisor based on cluster size. > > > Diffs > - > > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java > 2f080e3 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/AbstractTimelineAggregator.java > f8ec516 > > ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/AbstractTimelineAggregatorTest.java > 21b9839 > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py > b8e2c45 > > Diff: https://reviews.apache.org/r/45447/diff/ > > > Testing > --- > > Manually tested on a large cluster. > > Unit tests pass. > > > Thanks, > > Aravindan Vijayan > >
Re: Review Request 45518: AMBARI-15638 : AMS Sum Calculation Incorrect
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45518/#review126949 --- ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/AggregatorUtils.java (line 46) <https://reviews.apache.org/r/45518/#comment190028> This is result in divide by zero error if the count is used fro division ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricClusterAggregatorSecond.java (line 242) <https://reviews.apache.org/r/45518/#comment190031> This should be skipped if the denomintor is not incremented. - Sid Wagle On March 31, 2016, 9:35 p.m., Aravindan Vijayan wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45518/ > --- > > (Updated March 31, 2016, 9:35 p.m.) > > > Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle. > > > Bugs: AMBARI-15638 > https://issues.apache.org/jira/browse/AMBARI-15638 > > > Repository: ambari > > > Description > --- > > Sum Calculation is incorrect when the time range is more than 2 hrs. > This issue affects all metrics that are queried with the "sum" aggregator > (with or without hostname being specified) for more than a 2hr time-range. > > > Diffs > - > > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/AggregatorUtils.java > ce79b6f > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricClusterAggregatorSecond.java > 9354bc3 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricReadHelper.java > 846ae92 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/PhoenixTransactSQL.java > e67a5b8 > > Diff: https://reviews.apache.org/r/45518/diff/ > > > Testing > --- > > Manually tested. Unit tests pass. > > > Thanks, > > Aravindan Vijayan > >
Re: Review Request 45518: AMBARI-15638 : AMS Sum Calculation Incorrect
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45518/#review127000 --- Ship it! Ship It! - Sid Wagle On April 5, 2016, 12:11 a.m., Aravindan Vijayan wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45518/ > --- > > (Updated April 5, 2016, 12:11 a.m.) > > > Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle. > > > Bugs: AMBARI-15638 > https://issues.apache.org/jira/browse/AMBARI-15638 > > > Repository: ambari > > > Description > --- > > Sum Calculation is incorrect when the time range is more than 2 hrs. > This issue affects all metrics that are queried with the "sum" aggregator > (with or without hostname being specified) for more than a 2hr time-range. > > > Diffs > - > > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/AggregatorUtils.java > 126b3a3 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricClusterAggregatorSecond.java > 67c463f > > Diff: https://reviews.apache.org/r/45518/diff/ > > > Testing > --- > > Manually tested. Unit tests pass. > > > Thanks, > > Aravindan Vijayan > >
Re: Review Request 45739: Create script to export AMS metrics and re-import into AMS to visualize using Grafana
> On April 5, 2016, 2:08 p.m., Dmytro Sen wrote: > > ambari-server/src/main/resources/scripts/export_ams_metrics.py, line 58 > > <https://reviews.apache.org/r/45739/diff/1/?file=1325925#file1325925line58> > > > > Can be https:// Since this is masking AMS endpoint it should be ok to not mimic https. Meant as a debugging tool with older Production AMS version and 2.2.2 dev/support AMS - Grafana. - Sid --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45739/#review127100 --- On April 5, 2016, 11:09 a.m., Andrew Onischuk wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45739/ > --- > > (Updated April 5, 2016, 11:09 a.m.) > > > Review request for Ambari, Dmytro Sen and Sid Wagle. > > > Bugs: AMBARI-15710 > https://issues.apache.org/jira/browse/AMBARI-15710 > > > Repository: ambari > > > Description > --- > > Need to integrate this into 1 script and add it to ambari scripts to be used > by any support / dev person. > > The output options should allow the result to be directly consumable by > grafana or allow import in to live AMS instance. > > > Diffs > - > > ambari-server/src/main/resources/scripts/export_ams_metrics.py PRE-CREATION > > Diff: https://reviews.apache.org/r/45739/diff/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Andrew Onischuk > >
Re: Review Request 45739: Create script to export AMS metrics and re-import into AMS to visualize using Grafana
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45739/#review127113 --- Ship it! Ship It! - Sid Wagle On April 5, 2016, 11:09 a.m., Andrew Onischuk wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45739/ > --- > > (Updated April 5, 2016, 11:09 a.m.) > > > Review request for Ambari, Dmytro Sen and Sid Wagle. > > > Bugs: AMBARI-15710 > https://issues.apache.org/jira/browse/AMBARI-15710 > > > Repository: ambari > > > Description > --- > > Need to integrate this into 1 script and add it to ambari scripts to be used > by any support / dev person. > > The output options should allow the result to be directly consumable by > grafana or allow import in to live AMS instance. > > > Diffs > - > > ambari-server/src/main/resources/scripts/export_ams_metrics.py PRE-CREATION > > Diff: https://reviews.apache.org/r/45739/diff/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Andrew Onischuk > >
Re: Review Request 45772: AMBARI-15638 [AMS] Sum Calculation Incorrect (Patch 2)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45772/#review127209 --- Ship it! ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/AggregatorUtils.java (line 48) <https://reviews.apache.org/r/45772/#comment190380> Can values be null ? ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricClusterAggregatorSecond.java (line 245) <https://reviews.apache.org/r/45772/#comment190382> No need to braces. - Sid Wagle On April 5, 2016, 8:14 p.m., Aravindan Vijayan wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45772/ > --- > > (Updated April 5, 2016, 8:14 p.m.) > > > Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle. > > > Bugs: AMBARI-15638 > https://issues.apache.org/jira/browse/AMBARI-15638 > > > Repository: ambari > > > Description > --- > > 1. Including Zero values in METRIC_RECORD METRIC_COUNT. > 2. Fixing issue in TimelineMetricClusterAggregatorSecond > 3. Fixing issue in AVG() call in Phoenix query for higher level aggregates. > > > Diffs > - > > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/AggregatorUtils.java > 126b3a3 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricClusterAggregatorSecond.java > 67c463f > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/PhoenixTransactSQL.java > e3030f6 > > Diff: https://reviews.apache.org/r/45772/diff/ > > > Testing > --- > > Manually tested. > > > Thanks, > > Aravindan Vijayan > >
Re: Review Request 45845: AMBARI-15694 : AMS returns truncated results when it exceeds the metrics service default result limit config
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45845/#review127511 --- No unit test changes ? - Sid Wagle On April 6, 2016, 11:39 p.m., Aravindan Vijayan wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45845/ > --- > > (Updated April 6, 2016, 11:39 p.m.) > > > Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle. > > > Bugs: AMBARI-15694 > https://issues.apache.org/jira/browse/AMBARI-15694 > > > Repository: ambari > > > Description > --- > > When querying a large number of Hosts, AMS API returns a truncated set of > data-points. The number of data-points is > Metrics service default result limit / no. of hosts > > If the Metrics service default result limit is set to 15840 (current > default), then for 900 hosts, the data-points that will be returned for each > host will be 18. > > FIX > Take into account the hostnames requested also, while calculating calculating > result set size estimate. > > > Diffs > - > > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/PhoenixTransactSQL.java > 6a231c2 > > Diff: https://reviews.apache.org/r/45845/diff/ > > > Testing > --- > > Manually tested all charts in Ambari Web in all time ranges. > > Triggered exception message through Grafana. > > > Thanks, > > Aravindan Vijayan > >
Re: Review Request 45845: AMBARI-15694 : AMS returns truncated results when it exceeds the metrics service default result limit config
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45845/#review127512 --- Ship it! Ship It! - Sid Wagle On April 6, 2016, 11:39 p.m., Aravindan Vijayan wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45845/ > --- > > (Updated April 6, 2016, 11:39 p.m.) > > > Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle. > > > Bugs: AMBARI-15694 > https://issues.apache.org/jira/browse/AMBARI-15694 > > > Repository: ambari > > > Description > --- > > When querying a large number of Hosts, AMS API returns a truncated set of > data-points. The number of data-points is > Metrics service default result limit / no. of hosts > > If the Metrics service default result limit is set to 15840 (current > default), then for 900 hosts, the data-points that will be returned for each > host will be 18. > > FIX > Take into account the hostnames requested also, while calculating calculating > result set size estimate. > > > Diffs > - > > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/PhoenixTransactSQL.java > 6a231c2 > > Diff: https://reviews.apache.org/r/45845/diff/ > > > Testing > --- > > Manually tested all charts in Ambari Web in all time ranges. > > Triggered exception message through Grafana. > > > Thanks, > > Aravindan Vijayan > >
Review Request 45904: Ambari upgrade fails with MySQL DB and enforce gtid enabled
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45904/
---
Review request for Ambari, Jonathan Hurley, Myroslav Papirkovskyy, and Sumit
Mohanty.
Bugs: AMBARI-15774
https://issues.apache.org/jira/browse/AMBARI-15774
Repository: ambari
Description
---
Issue: Ambari upgrade is failing from 2.2.0 to 2.2.1.1 with below error:
{code}
ambari-server upgrade
Using python /usr/bin/python
Upgrading ambari-server
Updating properties in ambari.properties ...
WARNING: Can not find ambari.properties.rpmsave file from previous version,
skipping import of settings
WARNING: Can not find ambari-env.sh.rpmsave file from previous version,
skipping restore of environment settings
Fixing database objects owner
Ambari Server configured for MySQL. Confirm you have made a backup of the
Ambari Server database [y/n] (y)? y
Upgrading database schema
Error output from schema upgrade command:
Exception in thread "main" org.apache.ambari.server.AmbariException: Exception
[EclipseLink-4002] (Eclipse Persistence Services - 2.5.2.v20140319-9ad6abd):
org.eclipse.persistence.exceptions.DatabaseException
Internal Exception: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException:
Table 'ambari.TL_alert_notice' doesn't exist
Error Code: 1146
{code}
Diffs
-
ambari-server/src/main/java/org/apache/ambari/server/events/listeners/alerts/AlertReceivedListener.java
a3befa6
ambari-server/src/main/java/org/apache/ambari/server/events/listeners/alerts/AlertStateChangedListener.java
08563e3
ambari-server/src/main/java/org/apache/ambari/server/orm/dao/AlertDefinitionDAO.java
82fa48a
ambari-server/src/main/java/org/apache/ambari/server/orm/dao/AlertDispatchDAO.java
1f1aa45
ambari-server/src/main/java/org/apache/ambari/server/orm/dao/AlertsDAO.java
781d4cf
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/AlertCurrentEntity.java
604b00e
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/AlertHistoryEntity.java
03ffcde
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/AlertNoticeEntity.java
ae7495d
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/AlertResourceProviderTest.java
d611fe8
ambari-server/src/test/java/org/apache/ambari/server/orm/dao/AlertDefinitionDAOTest.java
36e75e7
ambari-server/src/test/java/org/apache/ambari/server/orm/dao/AlertsDAOTest.java
10f099e
ambari-server/src/test/java/org/apache/ambari/server/state/cluster/AlertDataManagerTest.java
e8ecd09
Diff: https://reviews.apache.org/r/45904/diff/
Testing
---
All alert unit test passed.
mvn clean test -Dtest=Alert* -Drat.ignoreErrors -DfailIfNoTests=false
Thanks,
Sid Wagle
Re: Review Request 45904: Ambari upgrade fails with MySQL DB and enforce gtid enabled
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45904/ --- (Updated April 7, 2016, 10:54 p.m.) Review request for Ambari, Jonathan Hurley, Myroslav Papirkovskyy, and Sumit Mohanty. Bugs: AMBARI-15774 https://issues.apache.org/jira/browse/AMBARI-15774 Repository: ambari Description (updated) --- Approach: - Added member varaibles only when underlying DB columns exist - Changed DELETE via implcit join query to find and delete - MySQL temp tables are only used for UPDATE ALL and DELETE ALL, so SELECT and DELETE is a safer approach - Visually inspected all Entities under org.apache.ambari.server.orm.entities.* for absence of this pattern Issue: Ambari upgrade is failing from 2.2.0 to 2.2.1.1 with below error: Error output from schema upgrade command: Exception in thread "main" org.apache.ambari.server.AmbariException: Exception [EclipseLink-4002] (Eclipse Persistence Services - 2.5.2.v20140319-9ad6abd): org.eclipse.persistence.exceptions.DatabaseException Internal Exception: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table 'ambari.TL_alert_notice' doesn't exist Error Code: 1146 Diffs - ambari-server/src/main/java/org/apache/ambari/server/events/listeners/alerts/AlertReceivedListener.java a3befa6 ambari-server/src/main/java/org/apache/ambari/server/events/listeners/alerts/AlertStateChangedListener.java 08563e3 ambari-server/src/main/java/org/apache/ambari/server/orm/dao/AlertDefinitionDAO.java 82fa48a ambari-server/src/main/java/org/apache/ambari/server/orm/dao/AlertDispatchDAO.java 1f1aa45 ambari-server/src/main/java/org/apache/ambari/server/orm/dao/AlertsDAO.java 781d4cf ambari-server/src/main/java/org/apache/ambari/server/orm/entities/AlertCurrentEntity.java 604b00e ambari-server/src/main/java/org/apache/ambari/server/orm/entities/AlertHistoryEntity.java 03ffcde ambari-server/src/main/java/org/apache/ambari/server/orm/entities/AlertNoticeEntity.java ae7495d ambari-server/src/test/java/org/apache/ambari/server/controller/internal/AlertResourceProviderTest.java d611fe8 ambari-server/src/test/java/org/apache/ambari/server/orm/dao/AlertDefinitionDAOTest.java 36e75e7 ambari-server/src/test/java/org/apache/ambari/server/orm/dao/AlertsDAOTest.java 10f099e ambari-server/src/test/java/org/apache/ambari/server/state/cluster/AlertDataManagerTest.java e8ecd09 Diff: https://reviews.apache.org/r/45904/diff/ Testing --- All alert unit test passed. mvn clean test -Dtest=Alert* -Drat.ignoreErrors -DfailIfNoTests=false Thanks, Sid Wagle
Re: Review Request 45904: Ambari upgrade fails with MySQL DB and enforce gtid enabled
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45904/ --- (Updated April 7, 2016, 11:01 p.m.) Review request for Ambari, Jonathan Hurley, Myroslav Papirkovskyy, and Sumit Mohanty. Bugs: AMBARI-15774 https://issues.apache.org/jira/browse/AMBARI-15774 Repository: ambari Description --- Approach: - Added member varaibles only when underlying DB columns exist - Changed DELETE via implcit join query to find and delete - MySQL temp tables are only used for UPDATE ALL and DELETE ALL, so SELECT and DELETE is a safer approach - Visually inspected all Entities under org.apache.ambari.server.orm.entities.* for absence of this pattern Issue: Ambari upgrade is failing from 2.2.0 to 2.2.1.1 with below error: Error output from schema upgrade command: Exception in thread "main" org.apache.ambari.server.AmbariException: Exception [EclipseLink-4002] (Eclipse Persistence Services - 2.5.2.v20140319-9ad6abd): org.eclipse.persistence.exceptions.DatabaseException Internal Exception: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table 'ambari.TL_alert_notice' doesn't exist Error Code: 1146 Diffs - ambari-server/src/main/java/org/apache/ambari/server/events/listeners/alerts/AlertReceivedListener.java a3befa6 ambari-server/src/main/java/org/apache/ambari/server/events/listeners/alerts/AlertStateChangedListener.java 08563e3 ambari-server/src/main/java/org/apache/ambari/server/orm/dao/AlertDefinitionDAO.java 82fa48a ambari-server/src/main/java/org/apache/ambari/server/orm/dao/AlertDispatchDAO.java 1f1aa45 ambari-server/src/main/java/org/apache/ambari/server/orm/dao/AlertsDAO.java 781d4cf ambari-server/src/main/java/org/apache/ambari/server/orm/entities/AlertCurrentEntity.java 604b00e ambari-server/src/main/java/org/apache/ambari/server/orm/entities/AlertHistoryEntity.java 03ffcde ambari-server/src/main/java/org/apache/ambari/server/orm/entities/AlertNoticeEntity.java ae7495d ambari-server/src/test/java/org/apache/ambari/server/controller/internal/AlertResourceProviderTest.java d611fe8 ambari-server/src/test/java/org/apache/ambari/server/orm/dao/AlertDefinitionDAOTest.java 36e75e7 ambari-server/src/test/java/org/apache/ambari/server/orm/dao/AlertsDAOTest.java 10f099e ambari-server/src/test/java/org/apache/ambari/server/state/cluster/AlertDataManagerTest.java e8ecd09 Diff: https://reviews.apache.org/r/45904/diff/ Testing (updated) --- Manually verified on 1 node repro cluster. All alert unit test passed. mvn clean test -Dtest=Alert* -Drat.ignoreErrors -DfailIfNoTests=false Thanks, Sid Wagle
Re: Review Request 45904: Ambari upgrade fails with MySQL DB and enforce gtid enabled
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45904/ --- (Updated April 7, 2016, 11:34 p.m.) Review request for Ambari, Jonathan Hurley, Myroslav Papirkovskyy, and Sumit Mohanty. Changes --- Remvoed some unnecessary setter calls. Bugs: AMBARI-15774 https://issues.apache.org/jira/browse/AMBARI-15774 Repository: ambari Description --- Approach: - Added member varaibles only when underlying DB columns exist - Changed DELETE via implcit join query to find and delete - MySQL temp tables are only used for UPDATE ALL and DELETE ALL, so SELECT and DELETE is a safer approach - Visually inspected all Entities under org.apache.ambari.server.orm.entities.* for absence of this pattern Issue: Ambari upgrade is failing from 2.2.0 to 2.2.1.1 with below error: Error output from schema upgrade command: Exception in thread "main" org.apache.ambari.server.AmbariException: Exception [EclipseLink-4002] (Eclipse Persistence Services - 2.5.2.v20140319-9ad6abd): org.eclipse.persistence.exceptions.DatabaseException Internal Exception: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table 'ambari.TL_alert_notice' doesn't exist Error Code: 1146 Diffs (updated) - ambari-server/src/main/java/org/apache/ambari/server/events/listeners/alerts/AlertReceivedListener.java a3befa6 ambari-server/src/main/java/org/apache/ambari/server/orm/dao/AlertDefinitionDAO.java 82fa48a ambari-server/src/main/java/org/apache/ambari/server/orm/dao/AlertDispatchDAO.java 1f1aa45 ambari-server/src/main/java/org/apache/ambari/server/orm/dao/AlertsDAO.java 781d4cf ambari-server/src/main/java/org/apache/ambari/server/orm/entities/AlertCurrentEntity.java 604b00e ambari-server/src/main/java/org/apache/ambari/server/orm/entities/AlertHistoryEntity.java 03ffcde ambari-server/src/main/java/org/apache/ambari/server/orm/entities/AlertNoticeEntity.java ae7495d ambari-server/src/test/java/org/apache/ambari/server/controller/internal/AlertResourceProviderTest.java d611fe8 ambari-server/src/test/java/org/apache/ambari/server/orm/dao/AlertDefinitionDAOTest.java 36e75e7 ambari-server/src/test/java/org/apache/ambari/server/orm/dao/AlertsDAOTest.java 10f099e Diff: https://reviews.apache.org/r/45904/diff/ Testing --- Manually verified on 1 node repro cluster. All alert unit test passed. mvn clean test -Dtest=Alert* -Drat.ignoreErrors -DfailIfNoTests=false Thanks, Sid Wagle
Re: Review Request 45846: AMBARI-15749 : Add AMS post processing function to show the metrics 'diff' over time.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45846/#review127694 --- Ship it! Ship It! - Sid Wagle On April 7, 2016, 11:38 p.m., Aravindan Vijayan wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45846/ > --- > > (Updated April 7, 2016, 11:38 p.m.) > > > Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle. > > > Bugs: AMBARI-15749 > https://issues.apache.org/jira/browse/AMBARI-15749 > > > Repository: ambari > > > Description > --- > > Introduce 'diff' post processing function to compute difference between > points across time. > > Also, removed the first point from the series during rate and diff > calculation. > > > Diffs > - > > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java > a5204e1 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/Function.java > 6f408a5 > > ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/FunctionTest.java > 46bc6f8 > > ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStoreTest.java > 512a7db > > Diff: https://reviews.apache.org/r/45846/diff/ > > > Testing > --- > > Manually tested. > > Added unit tests. > > > Thanks, > > Aravindan Vijayan > >
Re: Review Request 45845: AMBARI-15694 : AMS returns truncated results when it exceeds the metrics service default result limit config
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45845/#review127695 --- Ship it! Ship It! - Sid Wagle On April 7, 2016, 8:21 p.m., Aravindan Vijayan wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45845/ > --- > > (Updated April 7, 2016, 8:21 p.m.) > > > Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle. > > > Bugs: AMBARI-15694 > https://issues.apache.org/jira/browse/AMBARI-15694 > > > Repository: ambari > > > Description > --- > > When querying a large number of Hosts, AMS API returns a truncated set of > data-points. The number of data-points is > Metrics service default result limit / no. of hosts > > If the Metrics service default result limit is set to 15840 (current > default), then for 900 hosts, the data-points that will be returned for each > host will be 18. > > FIX > Take into account the hostnames requested also, while calculating calculating > result set size estimate. > > > Diffs > - > > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/PhoenixTransactSQL.java > c8cef27 > > ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TestPhoenixTransactSQL.java > 6bf15c7 > > Diff: https://reviews.apache.org/r/45845/diff/ > > > Testing > --- > > Manually tested all charts in Ambari Web in all time ranges. > > Triggered exception message through Grafana. > > > Thanks, > > Aravindan Vijayan > >
Re: Review Request 45873: Create a new alert type that is based on timeseries metrics
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45873/#review127697 --- Very nice pythonic changes to reduce verbosity (y) Why are we not chanigng the current script type alerts to use new scheme ? ambari-agent/src/main/python/ambari_agent/alerts/ams_alert.py (line 39) <https://reviews.apache.org/r/45873/#comment191087> Some documentation on what the calss does would be great. ambari-agent/src/main/python/ambari_agent/alerts/ams_alert.py (line 157) <https://reviews.apache.org/r/45873/#comment191086> OrderedDict support is only 2.7+ - Sid Wagle On April 7, 2016, 3:24 p.m., Dmytro Sen wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45873/ > --- > > (Updated April 7, 2016, 3:24 p.m.) > > > Review request for Ambari, Aravindan Vijayan, Jonathan Hurley, and Sid Wagle. > > > Bugs: AMBARI-15766 > https://issues.apache.org/jira/browse/AMBARI-15766 > > > Repository: ambari > > > Description > --- > > There are a few requirements around creating alerts based on the time-series > data stored in AMS. They fall in the the general category of: > - identify a metrics (name of the metrics, app name) > - identify a window (e.g. last 10 minutes, last 1 day) > - get the data points > - add them up, or > - compute average, or > - compute standard deviations, or > - ... > - compare the value to a threshold > - emit an alert if needed (WARN, CRITICAL, etc) > > We should also allow for alerts based on multiple metrics but there is no ask > for such a capability. Just something to keep in mind while designing. > It seems general enough that we should define an alert type that can be > instantiated by services to define specific alerts. Users can add additional > alerts if they so choose. > > > Diffs > - > > ambari-agent/src/main/python/ambari_agent/AlertSchedulerHandler.py b84832d > ambari-agent/src/main/python/ambari_agent/alerts/ams_alert.py PRE-CREATION > ambari-agent/src/main/python/ambari_agent/alerts/metric_alert.py d177bd4 > ambari-agent/src/test/python/ambari_agent/TestAlertSchedulerHandler.py > f4e7ba1 > ambari-agent/src/test/python/ambari_agent/TestAlerts.py e5f6a41 > ambari-agent/src/test/python/ambari_agent/TestAmsAlert.py PRE-CREATION > ambari-common/src/main/python/ambari_commons/aggregation_functions.py > PRE-CREATION > ambari-common/src/main/python/ambari_commons/firewall.py 64d396b > ambari-common/src/main/python/ambari_commons/urllib_handlers.py aa2e3f6 > > ambari-server/src/main/java/org/apache/ambari/server/state/alert/AlertDefinitionFactory.java > 1676b53 > > ambari-server/src/main/java/org/apache/ambari/server/state/alert/AmsSource.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/state/alert/MetricSource.java > 2ff438d > > ambari-server/src/main/java/org/apache/ambari/server/state/alert/SourceType.java > 357baf9 > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/alerts.json > 3612de2 > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/alerts/alert_metrics_deviation.py > 038592f > > ambari-server/src/test/java/org/apache/ambari/server/api/services/AmbariMetaInfoTest.java > 10b92d4 > > Diff: https://reviews.apache.org/r/45873/diff/ > > > Testing > --- > > Unit tests passed > > > Thanks, > > Dmytro Sen > >
Re: Review Request 45904: Ambari upgrade fails with MySQL DB and enforce gtid enabled
> On April 8, 2016, 3:36 a.m., Sumit Mohanty wrote: > > ambari-server/src/main/java/org/apache/ambari/server/orm/dao/AlertsDAO.java, > > line 745 > > <https://reviews.apache.org/r/45904/diff/2/?file=1331570#file1331570line745> > > > > Why this change? For compiler optimization??? Leftover debuggin artifact, I will edit before commitng or posting next update if needed. > On April 8, 2016, 3:36 a.m., Sumit Mohanty wrote: > > ambari-server/src/main/java/org/apache/ambari/server/orm/dao/AlertsDAO.java, > > line 938 > > <https://reviews.apache.org/r/45904/diff/2/?file=1331570#file1331570line938> > > > > Similar - why we are keeping count? Mostly debugging reasons. > On April 8, 2016, 3:36 a.m., Sumit Mohanty wrote: > > ambari-server/src/main/java/org/apache/ambari/server/orm/entities/AlertCurrentEntity.java, > > line 71 > > <https://reviews.apache.org/r/45904/diff/2/?file=1331571#file1331571line71> > > > > Oh! these existed - just not mapped to Entity obj Yes. - Sid --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45904/#review127728 --- On April 7, 2016, 11:34 p.m., Sid Wagle wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45904/ > --- > > (Updated April 7, 2016, 11:34 p.m.) > > > Review request for Ambari, Jonathan Hurley, Myroslav Papirkovskyy, and Sumit > Mohanty. > > > Bugs: AMBARI-15774 > https://issues.apache.org/jira/browse/AMBARI-15774 > > > Repository: ambari > > > Description > --- > > Approach: > > - Added member varaibles only when underlying DB columns exist > - Changed DELETE via implcit join query to find and delete > - MySQL temp tables are only used for UPDATE ALL and DELETE ALL, so SELECT > and DELETE is a safer approach > - Visually inspected all Entities under > org.apache.ambari.server.orm.entities.* for absence of this pattern > > Issue: Ambari upgrade is failing from 2.2.0 to 2.2.1.1 with below error: > > Error output from schema upgrade command: > Exception in thread "main" org.apache.ambari.server.AmbariException: > Exception [EclipseLink-4002] (Eclipse Persistence Services - > 2.5.2.v20140319-9ad6abd): > org.eclipse.persistence.exceptions.DatabaseException > Internal Exception: > com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table > 'ambari.TL_alert_notice' doesn't exist > Error Code: 1146 > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/events/listeners/alerts/AlertReceivedListener.java > a3befa6 > > ambari-server/src/main/java/org/apache/ambari/server/orm/dao/AlertDefinitionDAO.java > 82fa48a > > ambari-server/src/main/java/org/apache/ambari/server/orm/dao/AlertDispatchDAO.java > 1f1aa45 > ambari-server/src/main/java/org/apache/ambari/server/orm/dao/AlertsDAO.java > 781d4cf > > ambari-server/src/main/java/org/apache/ambari/server/orm/entities/AlertCurrentEntity.java > 604b00e > > ambari-server/src/main/java/org/apache/ambari/server/orm/entities/AlertHistoryEntity.java > 03ffcde > > ambari-server/src/main/java/org/apache/ambari/server/orm/entities/AlertNoticeEntity.java > ae7495d > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/AlertResourceProviderTest.java > d611fe8 > > ambari-server/src/test/java/org/apache/ambari/server/orm/dao/AlertDefinitionDAOTest.java > 36e75e7 > > ambari-server/src/test/java/org/apache/ambari/server/orm/dao/AlertsDAOTest.java > 10f099e > > Diff: https://reviews.apache.org/r/45904/diff/ > > > Testing > --- > > Manually verified on 1 node repro cluster. > > All alert unit test passed. > > mvn clean test -Dtest=Alert* -Drat.ignoreErrors -DfailIfNoTests=false > > > Thanks, > > Sid Wagle > >
Re: Review Request 45904: Ambari upgrade fails with MySQL DB and enforce gtid enabled
> On April 8, 2016, 12:26 p.m., Jonathan Hurley wrote: > > ambari-server/src/main/java/org/apache/ambari/server/orm/dao/AlertDispatchDAO.java, > > lines 687-692 > > <https://reviews.apache.org/r/45904/diff/2/?file=1331569#file1331569line687> > > > > This is a very dangerous query. This can literally return 10,000 > > entries. Thanks for pointing that out, I will explore an explicit join query as a workaround to offload it back to the DB. For the issue regarding reccurence of this situation I suggested adding gtid enforce policy in the system tests for the use case of delete cluster, etc. - Sid --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45904/#review127772 ----------- On April 7, 2016, 11:34 p.m., Sid Wagle wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45904/ > --- > > (Updated April 7, 2016, 11:34 p.m.) > > > Review request for Ambari, Jonathan Hurley, Myroslav Papirkovskyy, and Sumit > Mohanty. > > > Bugs: AMBARI-15774 > https://issues.apache.org/jira/browse/AMBARI-15774 > > > Repository: ambari > > > Description > --- > > Approach: > > - Added member varaibles only when underlying DB columns exist > - Changed DELETE via implcit join query to find and delete > - MySQL temp tables are only used for UPDATE ALL and DELETE ALL, so SELECT > and DELETE is a safer approach > - Visually inspected all Entities under > org.apache.ambari.server.orm.entities.* for absence of this pattern > > Issue: Ambari upgrade is failing from 2.2.0 to 2.2.1.1 with below error: > > Error output from schema upgrade command: > Exception in thread "main" org.apache.ambari.server.AmbariException: > Exception [EclipseLink-4002] (Eclipse Persistence Services - > 2.5.2.v20140319-9ad6abd): > org.eclipse.persistence.exceptions.DatabaseException > Internal Exception: > com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table > 'ambari.TL_alert_notice' doesn't exist > Error Code: 1146 > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/events/listeners/alerts/AlertReceivedListener.java > a3befa6 > > ambari-server/src/main/java/org/apache/ambari/server/orm/dao/AlertDefinitionDAO.java > 82fa48a > > ambari-server/src/main/java/org/apache/ambari/server/orm/dao/AlertDispatchDAO.java > 1f1aa45 > ambari-server/src/main/java/org/apache/ambari/server/orm/dao/AlertsDAO.java > 781d4cf > > ambari-server/src/main/java/org/apache/ambari/server/orm/entities/AlertCurrentEntity.java > 604b00e > > ambari-server/src/main/java/org/apache/ambari/server/orm/entities/AlertHistoryEntity.java > 03ffcde > > ambari-server/src/main/java/org/apache/ambari/server/orm/entities/AlertNoticeEntity.java > ae7495d > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/AlertResourceProviderTest.java > d611fe8 > > ambari-server/src/test/java/org/apache/ambari/server/orm/dao/AlertDefinitionDAOTest.java > 36e75e7 > > ambari-server/src/test/java/org/apache/ambari/server/orm/dao/AlertsDAOTest.java > 10f099e > > Diff: https://reviews.apache.org/r/45904/diff/ > > > Testing > --- > > Manually verified on 1 node repro cluster. > > All alert unit test passed. > > mvn clean test -Dtest=Alert* -Drat.ignoreErrors -DfailIfNoTests=false > > > Thanks, > > Sid Wagle > >
Re: Review Request 45873: Create a new alert type that is based on timeseries metrics
> On April 8, 2016, 12:18 p.m., Jonathan Hurley wrote: > > ambari-agent/src/main/python/ambari_agent/alerts/ams_alert.py, lines 66-70 > > <https://reviews.apache.org/r/45873/diff/1/?file=1330089#file1330089line66> > > > > Since this is contacting AMS, is there a better place to get this > > information from, like the AMS configurations? +1 Ambari server already has the cached location and port for collector(s), is there way to make that more of a first class citizen? - Sid --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45873/#review127769 --- On April 7, 2016, 3:24 p.m., Dmytro Sen wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45873/ > --- > > (Updated April 7, 2016, 3:24 p.m.) > > > Review request for Ambari, Aravindan Vijayan, Jonathan Hurley, and Sid Wagle. > > > Bugs: AMBARI-15766 > https://issues.apache.org/jira/browse/AMBARI-15766 > > > Repository: ambari > > > Description > --- > > There are a few requirements around creating alerts based on the time-series > data stored in AMS. They fall in the the general category of: > - identify a metrics (name of the metrics, app name) > - identify a window (e.g. last 10 minutes, last 1 day) > - get the data points > - add them up, or > - compute average, or > - compute standard deviations, or > - ... > - compare the value to a threshold > - emit an alert if needed (WARN, CRITICAL, etc) > > We should also allow for alerts based on multiple metrics but there is no ask > for such a capability. Just something to keep in mind while designing. > It seems general enough that we should define an alert type that can be > instantiated by services to define specific alerts. Users can add additional > alerts if they so choose. > > > Diffs > - > > ambari-agent/src/main/python/ambari_agent/AlertSchedulerHandler.py b84832d > ambari-agent/src/main/python/ambari_agent/alerts/ams_alert.py PRE-CREATION > ambari-agent/src/main/python/ambari_agent/alerts/metric_alert.py d177bd4 > ambari-agent/src/test/python/ambari_agent/TestAlertSchedulerHandler.py > f4e7ba1 > ambari-agent/src/test/python/ambari_agent/TestAlerts.py e5f6a41 > ambari-agent/src/test/python/ambari_agent/TestAmsAlert.py PRE-CREATION > ambari-common/src/main/python/ambari_commons/aggregation_functions.py > PRE-CREATION > ambari-common/src/main/python/ambari_commons/firewall.py 64d396b > ambari-common/src/main/python/ambari_commons/urllib_handlers.py aa2e3f6 > > ambari-server/src/main/java/org/apache/ambari/server/state/alert/AlertDefinitionFactory.java > 1676b53 > > ambari-server/src/main/java/org/apache/ambari/server/state/alert/AmsSource.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/state/alert/MetricSource.java > 2ff438d > > ambari-server/src/main/java/org/apache/ambari/server/state/alert/SourceType.java > 357baf9 > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/alerts.json > 3612de2 > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/alerts/alert_metrics_deviation.py > 038592f > > ambari-server/src/test/java/org/apache/ambari/server/api/services/AmbariMetaInfoTest.java > 10b92d4 > > Diff: https://reviews.apache.org/r/45873/diff/ > > > Testing > --- > > Unit tests passed > > > Thanks, > > Dmytro Sen > >
Re: Review Request 45904: Ambari upgrade fails with MySQL DB and enforce gtid enabled
> On April 8, 2016, 12:26 p.m., Jonathan Hurley wrote: > > ambari-server/src/main/java/org/apache/ambari/server/orm/dao/AlertDispatchDAO.java, > > lines 687-692 > > <https://reviews.apache.org/r/45904/diff/2/?file=1331569#file1331569line687> > > > > This is a very dangerous query. This can literally return 10,000 > > entries. > > Sid Wagle wrote: > Thanks for pointing that out, I will explore an explicit join query as a > workaround to offload it back to the DB. > For the issue regarding reccurence of this situation I suggested adding > gtid enforce policy in the system tests for the use case of delete cluster, > etc. Modified to reading ids only. - Sid --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45904/#review127772 ------- On April 8, 2016, 8:56 p.m., Sid Wagle wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45904/ > --- > > (Updated April 8, 2016, 8:56 p.m.) > > > Review request for Ambari, Jonathan Hurley, Myroslav Papirkovskyy, and Sumit > Mohanty. > > > Bugs: AMBARI-15774 > https://issues.apache.org/jira/browse/AMBARI-15774 > > > Repository: ambari > > > Description > --- > > Approach: > > - Added member varaibles only when underlying DB columns exist > - Changed DELETE via implcit join query to find and delete > - MySQL temp tables are only used for UPDATE ALL and DELETE ALL, so SELECT > and DELETE is a safer approach > - Visually inspected all Entities under > org.apache.ambari.server.orm.entities.* for absence of this pattern > > Issue: Ambari upgrade is failing from 2.2.0 to 2.2.1.1 with below error: > > Error output from schema upgrade command: > Exception in thread "main" org.apache.ambari.server.AmbariException: > Exception [EclipseLink-4002] (Eclipse Persistence Services - > 2.5.2.v20140319-9ad6abd): > org.eclipse.persistence.exceptions.DatabaseException > Internal Exception: > com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table > 'ambari.TL_alert_notice' doesn't exist > Error Code: 1146 > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/events/listeners/alerts/AlertReceivedListener.java > a3befa6 > > ambari-server/src/main/java/org/apache/ambari/server/orm/dao/AlertDefinitionDAO.java > 82fa48a > > ambari-server/src/main/java/org/apache/ambari/server/orm/dao/AlertDispatchDAO.java > 1f1aa45 > ambari-server/src/main/java/org/apache/ambari/server/orm/dao/AlertsDAO.java > 781d4cf > > ambari-server/src/main/java/org/apache/ambari/server/orm/entities/AlertCurrentEntity.java > 604b00e > > ambari-server/src/main/java/org/apache/ambari/server/orm/entities/AlertHistoryEntity.java > 03ffcde > > ambari-server/src/main/java/org/apache/ambari/server/orm/entities/AlertNoticeEntity.java > ae7495d > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/AlertResourceProviderTest.java > d611fe8 > > ambari-server/src/test/java/org/apache/ambari/server/orm/dao/AlertDefinitionDAOTest.java > 36e75e7 > > ambari-server/src/test/java/org/apache/ambari/server/orm/dao/AlertsDAOTest.java > 10f099e > > Diff: https://reviews.apache.org/r/45904/diff/ > > > Testing > --- > > Manually verified on 1 node repro cluster. > > All alert unit test passed. > > mvn clean test -Dtest=Alert* -Drat.ignoreErrors -DfailIfNoTests=false > > > Thanks, > > Sid Wagle > >
Re: Review Request 45904: Ambari upgrade fails with MySQL DB and enforce gtid enabled
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45904/ --- (Updated April 8, 2016, 8:56 p.m.) Review request for Ambari, Jonathan Hurley, Myroslav Papirkovskyy, and Sumit Mohanty. Changes --- - Instead of retrieving entity objects, changed to read only the ids - Adding batch delete for AleertCurrent and AlertNotice entities with defaut batch size 1 < Oracle IN clause limit - Added a unit test - All uni tests passed Bugs: AMBARI-15774 https://issues.apache.org/jira/browse/AMBARI-15774 Repository: ambari Description --- Approach: - Added member varaibles only when underlying DB columns exist - Changed DELETE via implcit join query to find and delete - MySQL temp tables are only used for UPDATE ALL and DELETE ALL, so SELECT and DELETE is a safer approach - Visually inspected all Entities under org.apache.ambari.server.orm.entities.* for absence of this pattern Issue: Ambari upgrade is failing from 2.2.0 to 2.2.1.1 with below error: Error output from schema upgrade command: Exception in thread "main" org.apache.ambari.server.AmbariException: Exception [EclipseLink-4002] (Eclipse Persistence Services - 2.5.2.v20140319-9ad6abd): org.eclipse.persistence.exceptions.DatabaseException Internal Exception: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table 'ambari.TL_alert_notice' doesn't exist Error Code: 1146 Diffs (updated) - ambari-server/src/main/java/org/apache/ambari/server/events/listeners/alerts/AlertReceivedListener.java a3befa6 ambari-server/src/main/java/org/apache/ambari/server/orm/dao/AlertDefinitionDAO.java 82fa48a ambari-server/src/main/java/org/apache/ambari/server/orm/dao/AlertDispatchDAO.java 1f1aa45 ambari-server/src/main/java/org/apache/ambari/server/orm/dao/AlertsDAO.java 781d4cf ambari-server/src/main/java/org/apache/ambari/server/orm/entities/AlertCurrentEntity.java 604b00e ambari-server/src/main/java/org/apache/ambari/server/orm/entities/AlertHistoryEntity.java 03ffcde ambari-server/src/main/java/org/apache/ambari/server/orm/entities/AlertNoticeEntity.java ae7495d ambari-server/src/test/java/org/apache/ambari/server/controller/internal/AlertResourceProviderTest.java d611fe8 ambari-server/src/test/java/org/apache/ambari/server/orm/dao/AlertDefinitionDAOTest.java 36e75e7 ambari-server/src/test/java/org/apache/ambari/server/orm/dao/AlertsDAOTest.java 10f099e Diff: https://reviews.apache.org/r/45904/diff/ Testing --- Manually verified on 1 node repro cluster. All alert unit test passed. mvn clean test -Dtest=Alert* -Drat.ignoreErrors -DfailIfNoTests=false Thanks, Sid Wagle
Re: Review Request 45980: If initial NN start runs into problem while communicating with JN, restart does not work
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45980/#review127988 --- ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py (line 322) <https://reviews.apache.org/r/45980/#comment191388> Shouldn't the same logic apply for HA scenario ? - Sid Wagle On April 9, 2016, 12:34 p.m., Andrew Onischuk wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45980/ > --- > > (Updated April 9, 2016, 12:34 p.m.) > > > Review request for Ambari and Dmytro Sen. > > > Bugs: AMBARI-15789 > https://issues.apache.org/jira/browse/AMBARI-15789 > > > Repository: ambari > > > Description > --- > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py > 70e27a5 > > Diff: https://reviews.apache.org/r/45980/diff/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Andrew Onischuk > >
Re: Review Request 45980: If initial NN start runs into problem while communicating with JN, restart does not work
> On April 9, 2016, 4:15 p.m., Sid Wagle wrote: > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py, > > line 322 > > <https://reviews.apache.org/r/45980/diff/1/?file=1338258#file1338258line322> > > > > Shouldn't the same logic apply for HA scenario ? > > Andrew Onischuk wrote: > The code you pointed to is never used. > > Sumit Mohanty wrote: > The if-else section is bit longer than needed but we are now checking for > error only for HA. Did you mean should it be done in non-HA as well? > > The format_namenode() code needs some clean up. It does not seem that we > use "force=true" anywhere. In that case can we remove the code that is un-reachable? - Sid --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45980/#review127988 --- On April 9, 2016, 4:46 p.m., Andrew Onischuk wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45980/ > ------- > > (Updated April 9, 2016, 4:46 p.m.) > > > Review request for Ambari, Dmytro Sen and Sid Wagle. > > > Bugs: AMBARI-15789 > https://issues.apache.org/jira/browse/AMBARI-15789 > > > Repository: ambari > > > Description > --- > > . > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py > 70e27a5 > > Diff: https://reviews.apache.org/r/45980/diff/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Andrew Onischuk > >
Re: Review Request 45980: If initial NN start runs into problem while communicating with JN, restart does not work
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45980/#review128010 --- Ship it! Ship It! - Sid Wagle On April 9, 2016, 4:46 p.m., Andrew Onischuk wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45980/ > --- > > (Updated April 9, 2016, 4:46 p.m.) > > > Review request for Ambari, Dmytro Sen and Sid Wagle. > > > Bugs: AMBARI-15789 > https://issues.apache.org/jira/browse/AMBARI-15789 > > > Repository: ambari > > > Description > --- > > . > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py > 70e27a5 > > Diff: https://reviews.apache.org/r/45980/diff/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Andrew Onischuk > >
Re: Review Request 45873: Create a new alert type that is based on timeseries metrics
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45873/#review128140 --- Ship it! Ship It! - Sid Wagle On April 11, 2016, 3:18 p.m., Dmytro Sen wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45873/ > --- > > (Updated April 11, 2016, 3:18 p.m.) > > > Review request for Ambari, Aravindan Vijayan, Jonathan Hurley, and Sid Wagle. > > > Bugs: AMBARI-15766 > https://issues.apache.org/jira/browse/AMBARI-15766 > > > Repository: ambari > > > Description > --- > > There are a few requirements around creating alerts based on the time-series > data stored in AMS. They fall in the the general category of: > - identify a metrics (name of the metrics, app name) > - identify a window (e.g. last 10 minutes, last 1 day) > - get the data points > - add them up, or > - compute average, or > - compute standard deviations, or > - ... > - compare the value to a threshold > - emit an alert if needed (WARN, CRITICAL, etc) > > We should also allow for alerts based on multiple metrics but there is no ask > for such a capability. Just something to keep in mind while designing. > It seems general enough that we should define an alert type that can be > instantiated by services to define specific alerts. Users can add additional > alerts if they so choose. > > > Diffs > - > > ambari-agent/src/main/python/ambari_agent/AlertSchedulerHandler.py b84832d > ambari-agent/src/main/python/ambari_agent/alerts/ams_alert.py PRE-CREATION > ambari-agent/src/main/python/ambari_agent/alerts/metric_alert.py d177bd4 > ambari-agent/src/test/python/ambari_agent/TestAlertSchedulerHandler.py > f4e7ba1 > ambari-agent/src/test/python/ambari_agent/TestAlerts.py 9caee8a > ambari-agent/src/test/python/ambari_agent/TestAmsAlert.py PRE-CREATION > ambari-common/src/main/python/ambari_commons/aggregate_functions.py > PRE-CREATION > ambari-common/src/main/python/ambari_commons/firewall.py 64d396b > ambari-common/src/main/python/ambari_commons/urllib_handlers.py aa2e3f6 > > ambari-server/src/main/java/org/apache/ambari/server/state/alert/AlertDefinitionFactory.java > 1676b53 > > ambari-server/src/main/java/org/apache/ambari/server/state/alert/AmsSource.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/state/alert/MetricSource.java > 2ff438d > > ambari-server/src/main/java/org/apache/ambari/server/state/alert/SourceType.java > 357baf9 > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/alerts.json > 3612de2 > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/alerts/alert_metrics_deviation.py > 038592f > > ambari-server/src/test/java/org/apache/ambari/server/api/services/AmbariMetaInfoTest.java > 5dc7f3b > > Diff: https://reviews.apache.org/r/45873/diff/ > > > Testing > --- > > Unit tests passed > > > Thanks, > > Dmytro Sen > >
Re: Review Request 46032: Restarting ambari-server after successful blueprint deploy of large cluster makes it unresponsive
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46032/#review128142 --- ambari-server/src/main/java/org/apache/ambari/server/topology/LogicalRequest.java (line 412) <https://reviews.apache.org/r/46032/#comment191509> Any chance of an NPE here ? - Sid Wagle On April 11, 2016, 3:15 p.m., Sebastian Toader wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/46032/ > --- > > (Updated April 11, 2016, 3:15 p.m.) > > > Review request for Ambari, Daniel Gergely, Laszlo Puskas, Robert Levas, > Sandor Magyari, Srimanth Gunturi, and Sid Wagle. > > > Bugs: AMBARI-15803 > https://issues.apache.org/jira/browse/AMBARI-15803 > > > Repository: ambari > > > Description > --- > > After restart Ambari lazily loads persisted cluster state from database in > order to figure out if there is anything pending for finalizing the cluster > creation > using Blueprints. The persisted host requests that don't have a host assigned > yet (pending host request) the server has to assign hosts as they register > with the server. > > Due to bug the server was erroneously tracking which hosts to wait for to > assign to the pending persisted host requests. > This led to NPEs later in the process of initializing state process database. > Each host registration first checks if initialization from > persisted state completed if not that triggers the initialization. Since the > init was continuously failing it was re-triggered on each host > registration leading to the unresponsiveness of the server. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/topology/LogicalRequest.java > 82edbcf > > ambari-server/src/test/java/org/apache/ambari/server/topology/LogicalRequestTest.java > PRE-CREATION > > Diff: https://reviews.apache.org/r/46032/diff/ > > > Testing > --- > > Manual testing using cluster creation templates with host groups with > multiple hosts specified through fixed fqdn list and also host predicates. > > > Unit tests: > Results : > > Tests run: 3550, Failures: 0, Errors: 0, Skipped: 36 > > > Thanks, > > Sebastian Toader > >
Re: Review Request 45904: Ambari upgrade fails with MySQL DB and enforce gtid enabled
> On April 11, 2016, 12:36 p.m., Jonathan Hurley wrote: > > ambari-server/src/main/java/org/apache/ambari/server/orm/dao/AlertDispatchDAO.java, > > lines 701-708 > > <https://reviews.apache.org/r/45904/diff/3/?file=1337530#file1337530line701> > > > > This solution is still far from ideal. You'll be making something like > > 20,000 database queries and if even 1 fails, then the entire transaction is > > rolled back. > > > > This is why we went for a pure JPQL approach here; it was clean and > > didn't cause any weird problems. > > > > This also doesn't address the fact that nothing prevents future JPQL > > queries from causing a similar problem. > > > > Perhaps we need to consider a different approach. The issue was that > > MySQL was using temporary tables outside of a transaction. If we can solve > > that problem, we can leave the existing JPQL. > > Myroslav Papirkovskyy wrote: > This is not quite correct. Issue was that EclipseLink uses MySQL > temporary tables INSIDE of transactions. > Temp tables outside of transaction are supported even with GTID enabled. > > However disabling temp tables simply doesn't work. EclipseLink starts to > generate invalid queries for MySQL. And after quick look at sources it seems > that major modifications are required to fix this. > For MySQL temp tables are used to bypass limitations of subqueries in > delete/update statements. > > Jonathan Hurley wrote: > Ah, that stinks. Yeah, my hope was that telling MySQL to not use temp > tables would be the ticket. But if it starts forming bad queries, that's no > good either. > > This is quite bad in that we really can't leverage JPQL anymore if we say > we support GTID. I'm curious if running in the WARN mode would be a good > balances. This way, GTID will be enabled, but some Ambari queries which > violated GTID constraints can still be allowed to execute. > > The current approach seems very fragile and extremely hard to enforce > moving forward. Agree that we would entirely rely on system test to catch any queries in the future. Unclear why you thing the implementation is fragile. - Sid ------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45904/#review128100 --- On April 8, 2016, 8:56 p.m., Sid Wagle wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45904/ > --- > > (Updated April 8, 2016, 8:56 p.m.) > > > Review request for Ambari, Jonathan Hurley, Myroslav Papirkovskyy, and Sumit > Mohanty. > > > Bugs: AMBARI-15774 > https://issues.apache.org/jira/browse/AMBARI-15774 > > > Repository: ambari > > > Description > --- > > Approach: > > - Added member varaibles only when underlying DB columns exist > - Changed DELETE via implcit join query to find and delete > - MySQL temp tables are only used for UPDATE ALL and DELETE ALL, so SELECT > and DELETE is a safer approach > - Visually inspected all Entities under > org.apache.ambari.server.orm.entities.* for absence of this pattern > > Issue: Ambari upgrade is failing from 2.2.0 to 2.2.1.1 with below error: > > Error output from schema upgrade command: > Exception in thread "main" org.apache.ambari.server.AmbariException: > Exception [EclipseLink-4002] (Eclipse Persistence Services - > 2.5.2.v20140319-9ad6abd): > org.eclipse.persistence.exceptions.DatabaseException > Internal Exception: > com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table > 'ambari.TL_alert_notice' doesn't exist > Error Code: 1146 > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/events/listeners/alerts/AlertReceivedListener.java > a3befa6 > > ambari-server/src/main/java/org/apache/ambari/server/orm/dao/AlertDefinitionDAO.java > 82fa48a > > ambari-server/src/main/java/org/apache/ambari/server/orm/dao/AlertDispatchDAO.java > 1f1aa45 > ambari-server/src/main/java/org/apache/ambari/server/orm/dao/AlertsDAO.java > 781d4cf > > ambari-server/src/main/java/org/apache/ambari/server/orm/entities/AlertCurrentEntity.java > 604b00e > > ambari-server/src/main/java/org/apache/ambari/server/orm/entities/AlertHistoryEntity.java > 03ffcde > > ambari-server/src/main/jav
Re: Review Request 45904: Ambari upgrade fails with MySQL DB and enforce gtid enabled
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45904/ --- (Updated April 11, 2016, 6:41 p.m.) Review request for Ambari, Jonathan Hurley, Myroslav Papirkovskyy, and Sumit Mohanty. Changes --- - Added helpful logging messages - Added comment right where the queries were changed Bugs: AMBARI-15774 https://issues.apache.org/jira/browse/AMBARI-15774 Repository: ambari Description --- Approach: - Added member varaibles only when underlying DB columns exist - Changed DELETE via implcit join query to find and delete - MySQL temp tables are only used for UPDATE ALL and DELETE ALL, so SELECT and DELETE is a safer approach - Visually inspected all Entities under org.apache.ambari.server.orm.entities.* for absence of this pattern Issue: Ambari upgrade is failing from 2.2.0 to 2.2.1.1 with below error: Error output from schema upgrade command: Exception in thread "main" org.apache.ambari.server.AmbariException: Exception [EclipseLink-4002] (Eclipse Persistence Services - 2.5.2.v20140319-9ad6abd): org.eclipse.persistence.exceptions.DatabaseException Internal Exception: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table 'ambari.TL_alert_notice' doesn't exist Error Code: 1146 Diffs (updated) - ambari-server/src/main/java/org/apache/ambari/server/events/listeners/alerts/AlertReceivedListener.java a3befa6 ambari-server/src/main/java/org/apache/ambari/server/orm/dao/AlertDefinitionDAO.java 82fa48a ambari-server/src/main/java/org/apache/ambari/server/orm/dao/AlertDispatchDAO.java 1f1aa45 ambari-server/src/main/java/org/apache/ambari/server/orm/dao/AlertsDAO.java 781d4cf ambari-server/src/main/java/org/apache/ambari/server/orm/entities/AlertCurrentEntity.java 604b00e ambari-server/src/main/java/org/apache/ambari/server/orm/entities/AlertHistoryEntity.java 03ffcde ambari-server/src/main/java/org/apache/ambari/server/orm/entities/AlertNoticeEntity.java ae7495d ambari-server/src/test/java/org/apache/ambari/server/controller/internal/AlertResourceProviderTest.java d611fe8 ambari-server/src/test/java/org/apache/ambari/server/orm/dao/AlertDefinitionDAOTest.java 36e75e7 ambari-server/src/test/java/org/apache/ambari/server/orm/dao/AlertsDAOTest.java 10f099e Diff: https://reviews.apache.org/r/45904/diff/ Testing --- Manually verified on 1 node repro cluster. All alert unit test passed. mvn clean test -Dtest=Alert* -Drat.ignoreErrors -DfailIfNoTests=false Thanks, Sid Wagle
Re: Review Request 46047: AMBARI-15812 : Changes to widgets to incorporate "rate" function in AMS not done in Upgrade path.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46047/#review128258 --- Ship it! Ship It! - Sid Wagle On April 11, 2016, 9:32 p.m., Aravindan Vijayan wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/46047/ > --- > > (Updated April 11, 2016, 9:32 p.m.) > > > Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle. > > > Bugs: AMBARI-15812 > https://issues.apache.org/jira/browse/AMBARI-15812 > > > Repository: ambari > > > Description > --- > > HDFS, Yarn, HBase widgets that show Counter metrics were changed to use > "rate" aggregation function for AMS in AMBARI-15385. However, these changes > were not done in Upgrade path. > > Hence, these widgets will still request counter data as it is instead of as a > Rate metric (._rate). > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog222.java > fc0e6b2 > > ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog222Test.java > cc5a04d > > Diff: https://reviews.apache.org/r/46047/diff/ > > > Testing > --- > > UpgradeCatalog tests pass. > > Manual testing done. > > 11 Apr 2016 20:26:00,333 INFO [main] UpgradeCatalog222:430 - Updating HDFS > widget definition. > 11 Apr 2016 20:26:00,342 INFO [main] UpgradeCatalog222:519 - Updating > widget: HDFS Bytes Read > 11 Apr 2016 20:26:00,361 INFO [main] UpgradeCatalog222:519 - Updating > widget: HDFS Bytes Written > 11 Apr 2016 20:26:00,373 INFO [main] UpgradeCatalog222:519 - Updating > widget: DataNode Process Disk I/O Utilization > 11 Apr 2016 20:26:00,386 INFO [main] UpgradeCatalog222:519 - Updating > widget: DataNode Process Network I/O Utilization > 11 Apr 2016 20:26:00,395 INFO [main] UpgradeCatalog222:519 - Updating > widget: NameNode RPC > 11 Apr 2016 20:26:00,405 INFO [main] UpgradeCatalog222:519 - Updating > widget: NN Connection Load > 11 Apr 2016 20:26:00,415 INFO [main] UpgradeCatalog222:519 - Updating > widget: NameNode GC count > 11 Apr 2016 20:26:00,421 INFO [main] UpgradeCatalog222:519 - Updating > widget: NameNode GC time > 11 Apr 2016 20:26:00,432 INFO [main] UpgradeCatalog222:449 - Updating YARN > widget definition. > 11 Apr 2016 20:26:00,433 INFO [main] UpgradeCatalog222:519 - Updating > widget: Container Failures > 11 Apr 2016 20:26:00,440 INFO [main] UpgradeCatalog222:519 - Updating > widget: Container Failures > 11 Apr 2016 20:26:00,452 INFO [main] UpgradeCatalog222:519 - Updating > widget: App Failures > 11 Apr 2016 20:26:00,457 INFO [main] UpgradeCatalog222:468 - Updating HBASE > widget definition. > 11 Apr 2016 20:26:00,461 INFO [main] UpgradeCatalog222:519 - Updating > widget: Reads and Writes > 11 Apr 2016 20:26:00,471 INFO [main] UpgradeCatalog222:519 - Updating > widget: Blocked Updates > 11 Apr 2016 20:26:00,484 INFO [main] UpgradeCatalog222:601 - Update widget > definition for HDFS corrupted blocks metric > 11 Apr 2016 20:26:00,494 INFO [main] AbstractUpgradeCatalog:464 - Applying > configuration with tag 'version1460406360491' to cluster 'c1' > > > Thanks, > > Aravindan Vijayan > >
Re: Review Request 46037: AMBARI-15698 : Metrics is not showing data
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46037/#review128583 --- Ship it! Ship It! - Sid Wagle On April 12, 2016, 11:53 p.m., Aravindan Vijayan wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/46037/ > --- > > (Updated April 12, 2016, 11:53 p.m.) > > > Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle. > > > Bugs: AMBARI-15698 > https://issues.apache.org/jira/browse/AMBARI-15698 > > > Repository: ambari > > > Description > --- > > PROBLEM > Metrics with aggregation function do not show up on Ambari Web. > > BUG & FIX > API did not return any data because of issues in Property Provider and a bug > in AMS. > > > Diffs > - > > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java > 51cacf3 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricReadHelper.java > bec103a > > Diff: https://reviews.apache.org/r/46037/diff/ > > > Testing > --- > > Manually tested. > > Unit tests pass. > > [INFO] Reactor Summary: > [INFO] > [INFO] Ambari Main ... SUCCESS [10.569s] > [INFO] Apache Ambari Project POM . SUCCESS [0.032s] > [INFO] Ambari Web SUCCESS [42.410s] > [INFO] Ambari Views .. SUCCESS [2.469s] > [INFO] Ambari Admin View . SUCCESS [26.923s] > [INFO] ambari-metrics SUCCESS [1.148s] > [INFO] Ambari Metrics Common . SUCCESS [2.506s] > [INFO] Ambari Metrics Hadoop Sink SUCCESS [5.348s] > [INFO] Ambari Metrics Flume Sink . SUCCESS [2.694s] > [INFO] Ambari Metrics Kafka Sink . SUCCESS [4.014s] > [INFO] Ambari Metrics Storm Sink . SUCCESS [1.381s] > [INFO] Ambari Metrics Collector .. SUCCESS [2:18.952s] > [INFO] Ambari Metrics Monitor SUCCESS [3.226s] > [INFO] Ambari Metrics Grafana SUCCESS [5.796s] > [INFO] Ambari Metrics Assembly ... SUCCESS [14.233s] > [INFO] Ambari Server . SUCCESS > [58:44.785s] > [INFO] Ambari Agent .. SUCCESS [9.778s] > [INFO] Ambari Client . SUCCESS [0.062s] > [INFO] Ambari Python Client .. SUCCESS [0.379s] > [INFO] Ambari Groovy Client .. SUCCESS [10.368s] > [INFO] Ambari Shell .. SUCCESS [0.041s] > [INFO] Ambari Python Shell ... SUCCESS [0.036s] > [INFO] Ambari Groovy Shell ... SUCCESS [7.090s] > [INFO] > > [INFO] BUILD SUCCESS > [INFO] > > [INFO] Total time: 1:03:35.031s > [INFO] Finished at: Mon Apr 11 11:03:19 PDT 2016 > [INFO] Final Memory: 102M/1015M > [INFO] > > > > Thanks, > > Aravindan Vijayan > >
Re: Review Request 46178: AMBARI-15872 : Provide ability to skip WAL file writes for AMS HBase tables
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46178/#review128792 --- Ship it! Ship It! - Sid Wagle On April 13, 2016, 10:52 p.m., Aravindan Vijayan wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/46178/ > --- > > (Updated April 13, 2016, 10:52 p.m.) > > > Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle. > > > Bugs: AMBARI-15872 > https://issues.apache.org/jira/browse/AMBARI-15872 > > > Repository: ambari > > > Description > --- > > For large clusters, this is a very useful optimization that can be employed. > > Added an optional config (Not available through default from the stack) > > ams-site : timeline.metrics.tables.durability > Valid values it can take : SKIP_WAL / SYNC_WAL / ASYNC_WAL / FSYNC_WAL > > > Diffs > - > > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java > 813387f > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricConfiguration.java > 0ebc248 > > ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/ITPhoenixHBaseAccessor.java > 89f3fbe > > Diff: https://reviews.apache.org/r/46178/diff/ > > > Testing > --- > > Manually tested. > > Added unit test. > > > Thanks, > > Aravindan Vijayan > >
Re: Review Request 46219: AMBARI-15892 : Incorrect (Negative) values are shown for memory metrics
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46219/#review128967 --- Ship it! Ship It! - Sid Wagle On April 14, 2016, 7:40 p.m., Aravindan Vijayan wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/46219/ > --- > > (Updated April 14, 2016, 7:40 p.m.) > > > Review request for Ambari, Andrii Tkach, Dmytro Sen, Jaimin Jetly, Sumit > Mohanty, Sid Wagle, and Yusaku Sako. > > > Bugs: AMBARI-15892 > https://issues.apache.org/jira/browse/AMBARI-15892 > > > Repository: ambari > > > Description > --- > > Issue > In the "NameNode HostLoad" graph, the negative values are seen for the > computed metric "Memory Utilization" which goes by the formula : > ( mem_total - (mem_free + mem_cache) ) *100 / mem_total > > Bug > AMBARI-15448 changed the way memory metrics are being reported to AMS. This > lead to a double subtraction of mem_cached, thereby leading to a negative > value intermittently. > > Fix > Change the widget to : > ( mem_total - mem_free ) *100 / mem_total > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog222.java > 0476326 > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/widgets.json > 0f8f105 > ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/widgets.json > a19a112 > > Diff: https://reviews.apache.org/r/46219/diff/ > > > Testing > --- > > Unit tests pass. > > Manual widget testing done. > > Manual upgrade testing pending. > > > Thanks, > > Aravindan Vijayan > >
Re: Review Request 46218: AlertReceivedListenerTest is Flaky When Running @Before Code
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46218/#review128968
---
Ship it!
Ship It!
- Sid Wagle
On April 14, 2016, 7:27 p.m., Jonathan Hurley wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46218/
> ---
>
> (Updated April 14, 2016, 7:27 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Nate Cole, and Sid Wagle.
>
>
> Bugs: AMBARI-15895
> https://issues.apache.org/jira/browse/AMBARI-15895
>
>
> Repository: ambari
>
>
> Description
> ---
>
> The {{AlertReceivedListenerTest}} occasionally fails with:
>
> ```
> javax.persistence.PersistenceException:
> Exception [EclipseLink-4002] (Eclipse Persistence Services -
> 2.6.2.v20151217-774c696): org.eclipse.persistence.exceptions.DatabaseException
> Internal Exception: java.sql.SQLIntegrityConstraintViolationException: The
> statement was aborted because it would have caused a duplicate key value in a
> unique or primary key constraint or unique index identified by
> 'UNQ_ALERT_GROUP_0' defined on 'ALERT_GROUP'.
> Error Code: 2
> Call: INSERT INTO alert_group (group_id, cluster_id, group_name, is_default,
> service_name) VALUES (?, ?, ?, ?, ?)
> bind => [5 parameters bound]
> at
> org.apache.ambari.server.state.alerts.AlertReceivedListenerTest.setup(AlertReceivedListenerTest.java:120)
> Caused by: org.eclipse.persistence.exceptions.DatabaseException:
>
> Internal Exception: java.sql.SQLIntegrityConstraintViolationException: The
> statement was aborted because it would have caused a duplicate key value in a
> unique or primary key constraint or unique index identified by
> 'UNQ_ALERT_GROUP_0' defined on 'ALERT_GROUP'.
> Error Code: 2
> Call: INSERT INTO alert_group (group_id, cluster_id, group_name, is_default,
> service_name) VALUES (?, ?, ?, ?, ?)
> bind => [5 parameters bound]
> at
> org.apache.ambari.server.state.alerts.AlertReceivedListenerTest.setup(AlertReceivedListenerTest.java:120)
> Caused by: java.sql.SQLIntegrityConstraintViolationException: The statement
> was aborted because it would have caused a duplicate key value in a unique or
> primary key constraint or unique index identified by 'UNQ_ALERT_GROUP_0'
> defined on 'ALERT_GROUP'.
> at
> org.apache.ambari.server.state.alerts.AlertReceivedListenerTest.setup(AlertReceivedListenerTest.java:120)
> Caused by: org.apache.derby.impl.jdbc.EmbedSQLException: The statement was
> aborted because it would have caused a duplicate key value in a unique or
> primary key constraint or unique index identified by 'UNQ_ALERT_GROUP_0'
> defined on 'ALERT_GROUP'.
> at
> org.apache.ambari.server.state.alerts.AlertReceivedListenerTest.setup(AlertReceivedListenerTest.java:120)
> Caused by: org.apache.derby.iapi.error.StandardException: The statement was
> aborted because it would have caused a duplicate key value in a unique or
> primary key constraint or unique index identified by 'UNQ_ALERT_GROUP_0'
> defined on 'ALERT_GROUP'.
> at
> org.apache.ambari.server.state.alerts.AlertReceivedListenerTest.setup(AlertReceivedListenerTest.java:120)
> ```
>
>
> Diffs
> -
>
> ambari-project/pom.xml 21c5baf
>
> ambari-server/src/test/java/org/apache/ambari/server/state/alerts/AlertReceivedListenerTest.java
> 28aa19a
>
> Diff: https://reviews.apache.org/r/46218/diff/
>
>
> Testing
> ---
>
> mvn clean test
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Review Request 46238: Refactor Metadata manager for supporting distributed collector
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46238/ --- Review request for Ambari, Aravindan Vijayan, Dmytro Sen, and Sumit Mohanty. Bugs: AMBARI-15902 https://issues.apache.org/jira/browse/AMBARI-15902 Repository: ambari Description --- Presently metadata is served from a write through cache, we need a bounded cache that gets refreshed periodically to allow sharded writes to succeed. Additionally the metadata cache needs ability to skip certain metrics, example: YARN Container metrics with too many unique names using a config. Diffs - ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricConfiguration.java 5b517ef ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/discovery/TimelineMetricMetadataManager.java 8e58203 ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/discovery/TimelineMetricMetadataSync.java 54ea200 ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/discovery/TestMetadataManager.java bf887be ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/discovery/TestMetadataSync.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java ddaff84 ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml e5758bf ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/templates/hadoop-metrics2.properties.j2 6f32000 Diff: https://reviews.apache.org/r/46238/diff/ Testing --- All metrics Unit tests passed. Thanks, Sid Wagle
Re: Review Request 46238: Refactor Metadata manager for supporting distributed collector
> On April 15, 2016, 8:19 a.m., Dmytro Sen wrote: > > ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/templates/hadoop-metrics2.properties.j2, > > line 104 > > <https://reviews.apache.org/r/46238/diff/1/?file=1345966#file1345966line104> > > > > It'll increase number of metrics sent from sinks to ams collector, so > > increase the collector load The ContainerMetrics will be syphoned off by HadoopMetricsSync into separate tables using API. This is currently scheduled for drop into 2.4.0, hence this is related change that we need to make so that the ContainerMetrics are not dropped. - Sid --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46238/#review129079 ----------- On April 14, 2016, 11:54 p.m., Sid Wagle wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/46238/ > --- > > (Updated April 14, 2016, 11:54 p.m.) > > > Review request for Ambari, Aravindan Vijayan, Dmytro Sen, and Sumit Mohanty. > > > Bugs: AMBARI-15902 > https://issues.apache.org/jira/browse/AMBARI-15902 > > > Repository: ambari > > > Description > --- > > Presently metadata is served from a write through cache, we need a bounded > cache that gets refreshed periodically to allow sharded writes to succeed. > > Additionally the metadata cache needs ability to skip certain metrics, > example: YARN Container metrics with too many unique names using a config. > > > Diffs > - > > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricConfiguration.java > 5b517ef > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/discovery/TimelineMetricMetadataManager.java > 8e58203 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/discovery/TimelineMetricMetadataSync.java > 54ea200 > > ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/discovery/TestMetadataManager.java > bf887be > > ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/discovery/TestMetadataSync.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java > ddaff84 > > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml > e5758bf > > ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/templates/hadoop-metrics2.properties.j2 > 6f32000 > > Diff: https://reviews.apache.org/r/46238/diff/ > > > Testing > --- > > All metrics Unit tests passed. > > > Thanks, > > Sid Wagle > >
Review Request 46274: Metrics Unit Tests Failing on b.a.o
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46274/
---
Review request for Ambari, Aravindan Vijayan, Jonathan Hurley, and Sumit
Mohanty.
Bugs: AMBARI-15923
https://issues.apache.org/jira/browse/AMBARI-15923
Repository: ambari
Description
---
https://builds.apache.org/job/Ambari-trunk-Commit/4662/#showFailuresLink
{code}
org.apache.hadoop.yarn.server.applicationhistoryservice.metrics.timeline.availability.TimelineMetricHAControllerTest.testHAControllerDistributedAggregation
org.apache.hadoop.yarn.server.applicationhistoryservice.metrics.timeline.availability.TimelineMetricHAControllerTest.testClusterOK
{code}
https://builds.apache.org/job/Ambari-branch-2.2/636/
{code}
org.apache.hadoop.yarn.server.applicationhistoryservice.metrics.timeline.discovery.TestMetadataManager.testSaveMetricsMetadata
org.apache.hadoop.yarn.server.applicationhistoryservice.metrics.timeline.discovery.TestMetadataManager.testClusterOK
{code}
Diffs
-
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/discovery/TimelineMetricMetadataManager.java
a39fcb6
ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/AbstractMiniHBaseClusterTest.java
7410e9d
ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/ITPhoenixHBaseAccessor.java
e6a360a
ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/availability/TimelineMetricHAControllerTest.java
04e8909
ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/discovery/TestMetadataManager.java
2b33cb95
Diff: https://reviews.apache.org/r/46274/diff/
Testing
---
Unit tests pass.
Thanks,
Sid Wagle
Re: Review Request 46137: AMBARI-15814: Enable debugging of ambari-server upgrade command
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46137/#review129369 --- ambari-server/src/main/python/ambari_server/serverUpgrade.py (line 67) <https://reviews.apache.org/r/46137/#comment192809> It should not take long enough for upgrade to execute that will allow remote debugging, don't you want a configurable supend action so JVM will pause sufficiently ? - Sid Wagle On April 18, 2016, 5:14 p.m., Nahappan Somasundaram wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/46137/ > --- > > (Updated April 18, 2016, 5:14 p.m.) > > > Review request for Ambari, Ajit Kumar, Jonathan Hurley, Sumit Mohanty, and > Sid Wagle. > > > Bugs: AMBARI-15814 > https://issues.apache.org/jira/browse/AMBARI-15814 > > > Repository: ambari > > > Description > --- > > AMBARI-15814: Enable debugging of ambari-server upgrade command > > ** Issue **: > Right now **ambari-server upgrade** cannot be debugged remotely. To enable > remote debugging, handle the -g/--debug flag the same way **ambari-server > start -g** works. > > ** Fix **: > Create a debug command line for JVM to execute, the same way as ambari-server > start -g works. > > > Diffs > - > > ambari-server/src/main/python/ambari_server/serverUpgrade.py 72c4185 > > Diff: https://reviews.apache.org/r/46137/diff/ > > > Testing > --- > > ** 1. mvn clean install ** > > [INFO] > > [INFO] Reactor Summary: > [INFO] > [INFO] Ambari Main ... SUCCESS [7.946s] > [INFO] Apache Ambari Project POM . SUCCESS [0.036s] > [INFO] Ambari Web SUCCESS [39.154s] > [INFO] Ambari Views .. SUCCESS [1.073s] > [INFO] Ambari Admin View . SUCCESS [8.779s] > [INFO] ambari-metrics SUCCESS [0.356s] > [INFO] Ambari Metrics Common . SUCCESS [0.467s] > [INFO] Ambari Metrics Hadoop Sink SUCCESS [1.190s] > [INFO] Ambari Metrics Flume Sink . SUCCESS [0.633s] > [INFO] Ambari Metrics Kafka Sink . SUCCESS [0.645s] > [INFO] Ambari Metrics Storm Sink . SUCCESS [1.640s] > [INFO] Ambari Metrics Collector .. SUCCESS [6.663s] > [INFO] Ambari Metrics Monitor SUCCESS [2.831s] > [INFO] Ambari Metrics Grafana SUCCESS [0.896s] > [INFO] Ambari Metrics Assembly ... SUCCESS [1:18.645s] > [INFO] Ambari Server . SUCCESS [2:49.890s] > [INFO] Ambari Functional Tests ... SUCCESS [2.347s] > [INFO] Ambari Agent .. SUCCESS [24.663s] > [INFO] Ambari Client . SUCCESS [0.045s] > [INFO] Ambari Python Client .. SUCCESS [0.983s] > [INFO] Ambari Groovy Client .. SUCCESS [2.203s] > [INFO] Ambari Shell .. SUCCESS [0.041s] > [INFO] Ambari Python Shell ... SUCCESS [0.720s] > [INFO] Ambari Groovy Shell ... SUCCESS [0.966s] > [INFO] ambari-logsearch .. SUCCESS [0.288s] > [INFO] Ambari Logsearch Portal ... SUCCESS [8.111s] > [INFO] Ambari Logsearch Log Feeder ... SUCCESS [3.025s] > [INFO] Ambari Logsearch Appender . SUCCESS [0.216s] > [INFO] Ambari Logsearch Assembly . SUCCESS [0.089s] > [INFO] > > [INFO] BUILD SUCCESS > [INFO] > > [INFO] Total time: 6:07.410s > [INFO] Finished at: Mon Apr 18 10:12:38 PDT 2016 > [INFO] Final Memory: 281M/1155M > [INFO] > > > ** 2. mvn test -DskipSurefireTests ** > > -- > Ran 260 tests in 6.521s > > OK > -
Re: Review Request 46137: AMBARI-15814: Enable debugging of ambari-server upgrade command
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46137/#review129382 --- Ship it! Ship It! - Sid Wagle On April 18, 2016, 5:39 p.m., Nahappan Somasundaram wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/46137/ > --- > > (Updated April 18, 2016, 5:39 p.m.) > > > Review request for Ambari, Ajit Kumar, Jonathan Hurley, Sumit Mohanty, and > Sid Wagle. > > > Bugs: AMBARI-15814 > https://issues.apache.org/jira/browse/AMBARI-15814 > > > Repository: ambari > > > Description > --- > > AMBARI-15814: Enable debugging of ambari-server upgrade command > > ** Issue **: > Right now **ambari-server upgrade** cannot be debugged remotely. To enable > remote debugging, handle the -g/--debug flag the same way **ambari-server > start -g** works. > > ** Fix **: > Create a debug command line for JVM to execute, the same way as ambari-server > start -g works. > > > Diffs > - > > ambari-server/src/main/python/ambari_server/serverUpgrade.py 72c4185 > > Diff: https://reviews.apache.org/r/46137/diff/ > > > Testing > --- > > ** 1. mvn clean install ** > > [INFO] > > [INFO] Reactor Summary: > [INFO] > [INFO] Ambari Main ... SUCCESS [7.946s] > [INFO] Apache Ambari Project POM . SUCCESS [0.036s] > [INFO] Ambari Web SUCCESS [39.154s] > [INFO] Ambari Views .. SUCCESS [1.073s] > [INFO] Ambari Admin View . SUCCESS [8.779s] > [INFO] ambari-metrics SUCCESS [0.356s] > [INFO] Ambari Metrics Common . SUCCESS [0.467s] > [INFO] Ambari Metrics Hadoop Sink SUCCESS [1.190s] > [INFO] Ambari Metrics Flume Sink . SUCCESS [0.633s] > [INFO] Ambari Metrics Kafka Sink . SUCCESS [0.645s] > [INFO] Ambari Metrics Storm Sink . SUCCESS [1.640s] > [INFO] Ambari Metrics Collector .. SUCCESS [6.663s] > [INFO] Ambari Metrics Monitor SUCCESS [2.831s] > [INFO] Ambari Metrics Grafana SUCCESS [0.896s] > [INFO] Ambari Metrics Assembly ... SUCCESS [1:18.645s] > [INFO] Ambari Server . SUCCESS [2:49.890s] > [INFO] Ambari Functional Tests ... SUCCESS [2.347s] > [INFO] Ambari Agent .. SUCCESS [24.663s] > [INFO] Ambari Client . SUCCESS [0.045s] > [INFO] Ambari Python Client .. SUCCESS [0.983s] > [INFO] Ambari Groovy Client .. SUCCESS [2.203s] > [INFO] Ambari Shell .. SUCCESS [0.041s] > [INFO] Ambari Python Shell ... SUCCESS [0.720s] > [INFO] Ambari Groovy Shell ... SUCCESS [0.966s] > [INFO] ambari-logsearch .. SUCCESS [0.288s] > [INFO] Ambari Logsearch Portal ... SUCCESS [8.111s] > [INFO] Ambari Logsearch Log Feeder ... SUCCESS [3.025s] > [INFO] Ambari Logsearch Appender . SUCCESS [0.216s] > [INFO] Ambari Logsearch Assembly . SUCCESS [0.089s] > [INFO] > > [INFO] BUILD SUCCESS > [INFO] > > [INFO] Total time: 6:07.410s > [INFO] Finished at: Mon Apr 18 10:12:38 PDT 2016 > [INFO] Final Memory: 281M/1155M > [INFO] > > > ** 2. mvn test -DskipSurefireTests ** > > -- > Ran 260 tests in 6.521s > > OK > -- > Total run:963 > Total errors:0 > Total failures:0 > OK > [INFO] > > [INFO] BUILD SUCCESS > [INFO] > ---
Re: Review Request 46342: BUG-55643 Add container metrics in AMS
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46342/#review129425 --- ambari-metrics/ambari-metrics-hadoop-sink/src/main/java/org/apache/hadoop/metrics2/sink/timeline/HadoopTimelineMetricsSink.java (line 354) <https://reviews.apache.org/r/46342/#comment192850> This Exception is dangerous to be thrown from the Sink is not handled by the caller. We should be logging the exceotion and moving on. - Sid Wagle On April 18, 2016, 7:04 p.m., Jian He wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/46342/ > --- > > (Updated April 18, 2016, 7:04 p.m.) > > > Review request for Ambari, Aravindan Vijayan, Sumit Mohanty, and Sid Wagle. > > > Repository: ambari > > > Description > --- > > BUG-55643 Add container metrics in AMS > > > Diffs > - > > > ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/AbstractTimelineMetricsSink.java > 28d3b9c > > ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/ContainerMetric.java > PRE-CREATION > > ambari-metrics/ambari-metrics-hadoop-sink/src/main/java/org/apache/hadoop/metrics2/sink/timeline/HadoopTimelineMetricsSink.java > db8791f > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java > ab11333 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java > e434d33 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricConfiguration.java > c0093a0 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricStore.java > 2f08f3f > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/PhoenixTransactSQL.java > 0efa68f > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/webapp/TimelineWebServices.java > d2526a0 > > ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/ITPhoenixHBaseAccessor.java > f61d619 > > ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TestTimelineMetricStore.java > 61d6e71 > > Diff: https://reviews.apache.org/r/46342/diff/ > > > Testing > --- > > Did end-to-end testing on cluster, NM can publish the container metrics, and > AMS will store that in phoenix > > > Thanks, > > Jian He > >
Re: Review Request 46342: BUG-55643 Add container metrics in AMS
> On April 18, 2016, 8:19 p.m., Aravindan Vijayan wrote: > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricConfiguration.java, > > line 57 > > <https://reviews.apache.org/r/46342/diff/1/?file=1348635#file1348635line57> > > > > This config needs to go into the stack > > (ambari/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml) > > for UI to pick it up. > > > > It should also go into the Upgrade path so that this config gets added > > to the service on Upgrade. Perhaps, we can use another JIRA to track that. > > Aravindan Vijayan wrote: > This issue is valid only if we need to control the TTL value of the > container metrics table through Ambari. AS long as ther is a valid default, it is ok to not add it to Ambari stack or the UpgradeCatalog. - Sid --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46342/#review129418 --- On April 18, 2016, 7:04 p.m., Jian He wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/46342/ > --- > > (Updated April 18, 2016, 7:04 p.m.) > > > Review request for Ambari, Aravindan Vijayan, Sumit Mohanty, and Sid Wagle. > > > Repository: ambari > > > Description > --- > > BUG-55643 Add container metrics in AMS > > > Diffs > - > > > ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/AbstractTimelineMetricsSink.java > 28d3b9c > > ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/ContainerMetric.java > PRE-CREATION > > ambari-metrics/ambari-metrics-hadoop-sink/src/main/java/org/apache/hadoop/metrics2/sink/timeline/HadoopTimelineMetricsSink.java > db8791f > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java > ab11333 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java > e434d33 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricConfiguration.java > c0093a0 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricStore.java > 2f08f3f > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/PhoenixTransactSQL.java > 0efa68f > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/webapp/TimelineWebServices.java > d2526a0 > > ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/ITPhoenixHBaseAccessor.java > f61d619 > > ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TestTimelineMetricStore.java > 61d6e71 > > Diff: https://reviews.apache.org/r/46342/diff/ > > > Testing > --- > > Did end-to-end testing on cluster, NM can publish the container metrics, and > AMS will store that in phoenix > > > Thanks, > > Jian He > >
Re: Review Request 46342: BUG-55643 Add container metrics in AMS
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46342/#review129445 --- Ship it! Ship It! - Sid Wagle On April 18, 2016, 10:24 p.m., Jian He wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/46342/ > --- > > (Updated April 18, 2016, 10:24 p.m.) > > > Review request for Ambari, Aravindan Vijayan, Sumit Mohanty, and Sid Wagle. > > > Repository: ambari > > > Description > --- > > BUG-55643 Add container metrics in AMS > > > Diffs > - > > > ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/AbstractTimelineMetricsSink.java > 28d3b9c > > ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/ContainerMetric.java > PRE-CREATION > > ambari-metrics/ambari-metrics-hadoop-sink/src/main/java/org/apache/hadoop/metrics2/sink/timeline/HadoopTimelineMetricsSink.java > db8791f > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java > ab11333 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java > e434d33 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricConfiguration.java > c0093a0 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricStore.java > 2f08f3f > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/PhoenixTransactSQL.java > 0efa68f > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/webapp/TimelineWebServices.java > d2526a0 > > ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/ITPhoenixHBaseAccessor.java > f61d619 > > ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TestTimelineMetricStore.java > 61d6e71 > > Diff: https://reviews.apache.org/r/46342/diff/ > > > Testing > --- > > Did end-to-end testing on cluster, NM can publish the container metrics, and > AMS will store that in phoenix > > > Thanks, > > Jian He > >
Review Request 46355: NameNode Last Checkpoint script alert definition does not trigger based on uncommitted transactions
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46355/ --- Review request for Ambari, Aravindan Vijayan, Dmytro Sen, Jonathan Hurley, and Sumit Mohanty. Bugs: AMBARI-15953 https://issues.apache.org/jira/browse/AMBARI-15953 Repository: ambari Description --- The 'NameNode Last Checkpoint' alert description says "This service-level alert will trigger if the last time that the NameNode performed a checkpoint was too long ago. It will also trigger if the number of uncommitted transactions is beyond a certain threshold." But the default alert definition seems to miss the threshold parameters for alerting the number of uncommitted transactions. Diffs - ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/alerts.json fc3e21f ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/alerts/alert_checkpoint_time.py 71e34e6 ambari-server/src/test/python/stacks/2.0.6/HDFS/test_alert_checkpoint_time.py PRE-CREATION Diff: https://reviews.apache.org/r/46355/diff/ Testing --- -- Ran 246 tests in 6.541s OK -- Total run:928 Total errors:0 Total failures:0 OK Thanks, Sid Wagle
Re: Review Request 46355: NameNode Last Checkpoint script alert definition does not trigger based on uncommitted transactions
> On April 19, 2016, 3:13 a.m., Jonathan Hurley wrote: > > Do you need to also set these new values on upgrade? I see that you put > > defaults in, but the thresholds will never show in existing configurations. Ahh dank missed upgrade, thanks for the reminder, will post updates tomorrow. - Sid --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46355/#review129475 --- On April 19, 2016, 12:26 a.m., Sid Wagle wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/46355/ > --- > > (Updated April 19, 2016, 12:26 a.m.) > > > Review request for Ambari, Aravindan Vijayan, Dmytro Sen, Jonathan Hurley, > and Sumit Mohanty. > > > Bugs: AMBARI-15953 > https://issues.apache.org/jira/browse/AMBARI-15953 > > > Repository: ambari > > > Description > --- > > The 'NameNode Last Checkpoint' alert description says "This service-level > alert will trigger if the last time that the NameNode performed a checkpoint > was too long ago. It will also trigger if the number of uncommitted > transactions is beyond a certain threshold." > > But the default alert definition seems to miss the threshold parameters for > alerting the number of uncommitted transactions. > > > Diffs > - > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/alerts.json > fc3e21f > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/alerts/alert_checkpoint_time.py > 71e34e6 > > ambari-server/src/test/python/stacks/2.0.6/HDFS/test_alert_checkpoint_time.py > PRE-CREATION > > Diff: https://reviews.apache.org/r/46355/diff/ > > > Testing > --- > > -- > Ran 246 tests in 6.541s > > OK > -- > Total run:928 > Total errors:0 > Total failures:0 > OK > > > Thanks, > > Sid Wagle > >
Re: Review Request 46355: NameNode Last Checkpoint script alert definition does not trigger based on uncommitted transactions
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46355/ --- (Updated April 19, 2016, 10:04 p.m.) Review request for Ambari, Aravindan Vijayan, Dmytro Sen, Jonathan Hurley, and Sumit Mohanty. Changes --- Modifications made to UpgradeCatalog240 Bugs: AMBARI-15953 https://issues.apache.org/jira/browse/AMBARI-15953 Repository: ambari Description --- The 'NameNode Last Checkpoint' alert description says "This service-level alert will trigger if the last time that the NameNode performed a checkpoint was too long ago. It will also trigger if the number of uncommitted transactions is beyond a certain threshold." But the default alert definition seems to miss the threshold parameters for alerting the number of uncommitted transactions. Diffs (updated) - ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java 097a079 ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/alerts.json 019fa60 ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/alerts/alert_checkpoint_time.py e165c7b ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog240Test.java e3e8d35 Diff: https://reviews.apache.org/r/46355/diff/ Testing --- -- Ran 246 tests in 6.541s OK -- Total run:928 Total errors:0 Total failures:0 OK Thanks, Sid Wagle
Re: Review Request 46413: AMBARI-15982 : AMS metrics gaps/inconsistencies in data
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46413/#review129634 --- Ship it! Ship It! - Sid Wagle On April 19, 2016, 10:08 p.m., Aravindan Vijayan wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/46413/ > --- > > (Updated April 19, 2016, 10:08 p.m.) > > > Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle. > > > Bugs: AMBARI-15982 > https://issues.apache.org/jira/browse/AMBARI-15982 > > > Repository: ambari > > > Description > --- > > PROBLEM > Gaps are being seen in the data shown in the Yarn Home Grafana dashboard. > > BUG > Changes that were made through AMBARI-15026 were not added to the Upgrade > path. Hence, the hadoop daemons reported data at 1 min intervals instead of > 10 seconds. > > FIX > Changse made to the Upgrade path for 2.4.0 > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java > 097a079 > > ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog240Test.java > e3e8d35 > > Diff: https://reviews.apache.org/r/46413/diff/ > > > Testing > --- > > Manually tested upgrade. > > > Thanks, > > Aravindan Vijayan > >
Re: Review Request 45739: Create script to export AMS metrics and re-import into AMS to visualize using Grafana
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45739/#review129742 --- Fix it, then Ship it! ambari-server/src/main/resources/scripts/export_ams_metrics.py (line 140) <https://reviews.apache.org/r/45739/#comment193285> Better to set seriesStartTime = valid timstamp like now. - Sid Wagle On April 20, 2016, 1:44 p.m., Andrew Onischuk wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45739/ > --- > > (Updated April 20, 2016, 1:44 p.m.) > > > Review request for Ambari, Dmytro Sen and Sid Wagle. > > > Bugs: AMBARI-15710 > https://issues.apache.org/jira/browse/AMBARI-15710 > > > Repository: ambari > > > Description > --- > > Need to integrate this into 1 script and add it to ambari scripts to be used > by any support / dev person. > > The output options should allow the result to be directly consumable by > grafana or allow import in to live AMS instance. > > > Diffs > - > > ambari-server/src/main/resources/scripts/export_ams_metrics.py 9344841 > > Diff: https://reviews.apache.org/r/45739/diff/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Andrew Onischuk > >
Re: Review Request 43926: Frameworks support for bulk delete API
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43926/#review129829
---
Ship it!
Ship It!
- Sid Wagle
On April 20, 2016, 6:45 p.m., Ajit Kumar wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43926/
> ---
>
> (Updated April 20, 2016, 6:45 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Zhe (Joe) Wang, Nate Cole,
> Sumit Mohanty, Sid Wagle, and Yusaku Sako.
>
>
> Bugs: AMBARI-15150
> https://issues.apache.org/jira/browse/AMBARI-15150
>
>
> Repository: ambari
>
>
> Description
> ---
>
> Currently DELETE API only returns status code and no body. This doesn't fit
> well in bulk delete scenario as user needs to know which keys were deleted
> and which one failed and why. For this, ResourceProvider needs to return
> metadata as part of RequestStatus object which can be translated to metadata
> for Result object at API layer.
>
> At API layer, for all failed keys, reason code (like 404) is required which
> will make scripting easy instead of just including error message as string.
> API response for bulk delete
> {code}
> curl -u admin:admin -H 'X-Requested-By: ambari' -X DELETE
> http://c6401.ambari.apache.org:8080/api/v1/clusters/a/hosts -d
> '{"RequestInfo":{"query":"Hosts/host_name.in(c6401.ambari.apache.org,c6402.ambari.apache.org)"}}'
> {
> "deleteResult" : [
> {
> "deleted" : {
> "key" : "c6402.ambari.apache.org"
> }
> },
> {
> "error" : {
> "key" : "c6401.ambari.apache.org",
> "code" : 500,
> "message" : "org.apache.ambari.server.AmbariException: Cannot remove
> host c6401.ambari.apache.org from a. The following roles exist, and these
> components must be stopped if running, and then deleted: ZOOKEEPER_SERVER"
> }
> }
> ]
> }
> {code}
>
>
> Sample code for how each resource provider can set metadata for delete
> https://reviews.apache.org/r/43927/.
>
>
> Diffs
> -
>
>
> ambari-server/src/main/java/org/apache/ambari/server/api/handlers/BaseManagementHandler.java
> c34f0d7
>
> ambari-server/src/main/java/org/apache/ambari/server/api/handlers/CreateHandler.java
> 691a3ee
>
> ambari-server/src/main/java/org/apache/ambari/server/api/handlers/DeleteHandler.java
> 8580196
>
> ambari-server/src/main/java/org/apache/ambari/server/api/handlers/QueryCreateHandler.java
> 534ee7b
>
> ambari-server/src/main/java/org/apache/ambari/server/api/handlers/UpdateHandler.java
> bb66fae
>
> ambari-server/src/main/java/org/apache/ambari/server/api/services/DeleteResultMetadata.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/api/services/Result.java
> fabab62
>
> ambari-server/src/main/java/org/apache/ambari/server/api/services/ResultImpl.java
> adcbde1
>
> ambari-server/src/main/java/org/apache/ambari/server/api/services/ResultMetadata.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/api/services/serializers/JsonSerializer.java
> c17f162
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractResourceProvider.java
> ac9935d
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/DeleteStatusMetaData.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RequestStatusImpl.java
> b0fde8a
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/spi/RequestStatus.java
> f9847a9
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/spi/RequestStatusMetaData.java
> PRE-CREATION
>
> ambari-server/src/test/java/org/apache/ambari/server/api/handlers/DeleteHandlerTest.java
> fb70be6
>
> ambari-server/src/test/java/org/apache/ambari/server/api/services/DeleteResultMetaDataTest.java
> PRE-CREATION
>
> ambari-server/src/test/java/org/apache/ambari/server/api/services/serializers/JsonSerializerTest.java
> d854dcf
>
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/DeleteStatusMetaDataTest.java
> PRE-CREATION
>
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/RequestStatusImplTest.java
> cd61822
>
> Diff: https://reviews.apache.org/r/43926/diff/
>
>
> Testing
> ---
>
> manual testing
> Unit test
>
>
> Thanks,
>
> Ajit Kumar
>
>
Re: Review Request 46500: Regenerating keytabs on re-imaged hosts results in error during 'Creating Principals'
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46500/#review129923
---
Ship it!
Ship It!
- Sid Wagle
On April 21, 2016, 5:01 p.m., Robert Levas wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46500/
> ---
>
> (Updated April 21, 2016, 5:01 p.m.)
>
>
> Review request for Ambari, Jonathan Hurley, Nate Cole, Oliver Szabo, Srimanth
> Gunturi, and Sid Wagle.
>
>
> Bugs: AMBARI-16009
> https://issues.apache.org/jira/browse/AMBARI-16009
>
>
> Repository: ambari
>
>
> Description
> ---
>
> We had a 1600 unsecured cluster initially, from which 700 nodes were
> destroyed. Though Ambari-server knew of 1600 hosts, only 900 were
> heartbeating. At this point we secured the cluster and everything was good.
> Then we brought back the 700 hosts, which started heartbeating with
> ambari-server.
>
> At this point we did 'Regenerate Keytabs' which failed at the 'Create
> Principals' step (image attached), as it was trying to re-create principal
> which is already existing with kadmin, and with ambari-server.
>
> #Create Principals
> Stderr:
> ```
> 2016-04-21 01:28:52,985 - Failed to create or update principal,
> HTTP/host1.example@example.com - Failed to create service principal for
> HTTP/host1.example@example.com
> STDOUT: Authenticating as principal admin/admin with password.
>
> STDERR: WARNING: no policy specified for HTTP/host1.example@example.com;
> defaulting to no policy
> add_principal: Principal or policy already exists while creating
> "HTTP/host1.example@example.com".
> {noformat}
>
> Stdout:
> {noformat}
> 2016-04-21 01:27:32,400 - Processing identities...
> 2016-04-21 01:28:29,874 - Processing principal,
> HTTP/host1.example@example.com
> ```
>
>
> Diffs
> -
>
>
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/MITKerberosOperationHandler.java
> f48052f
>
> ambari-server/src/test/java/org/apache/ambari/server/serveraction/kerberos/MITKerberosOperationHandlerTest.java
> d15db17
>
> Diff: https://reviews.apache.org/r/46500/diff/
>
>
> Testing
> ---
>
> Manually tested enabling Kerberos and Regenerating Keytabs both having
> unexpected principals in or removed from the KDC as needed to attempt to
> generate the issue.
>
> #Local test results:
> [INFO]
>
> [INFO] BUILD SUCCESS
> [INFO]
>
> [INFO] Total time: 1:19:08.563s
> [INFO] Finished at: Thu Apr 21 12:45:33 EDT 2016
> [INFO] Final Memory: 60M/1866M
> [INFO]
>
>
> # Jenkins test results: PENDING
>
>
> Thanks,
>
> Robert Levas
>
>
Re: Review Request 46496: Host_status stuck in UNKNOWN status after blueprint deploy with host in heartbeat-lost
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46496/#review129926 --- Ship it! Ship It! - Sid Wagle On April 21, 2016, 3:51 p.m., Sebastian Toader wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/46496/ > --- > > (Updated April 21, 2016, 3:51 p.m.) > > > Review request for Ambari, Daniel Gergely, Laszlo Puskas, Sandor Magyari, > Sumit Mohanty, and Sid Wagle. > > > Bugs: AMBARI-16013 > https://issues.apache.org/jira/browse/AMBARI-16013 > > > Repository: ambari > > > Description > --- > > When hosts register to Ambari server the `TopologyManager` adds these to its > `availableHosts` collection. When a cluster is provisioned using Blueprints > `TopologyManager` tries to allocate required hosts to hostgroups from the > available hosts collection. In case hosts turned into HEARTBEAT_LOST state > these were not removed from `availableHosts` this resulting scheduling > logical tasks to unreachable hosts. When these unreachable hosts become > available re-register with Ambari server. The server since already scheduled > logical tasks for these it won't try again thus will never create role > commands to be executed by the hosts. > > `TopologyManager` has been hooked now to the HEARTBEAT_LOST state transition > to remove the host in question from its internal `availableHosts` collection. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/state/host/HostImpl.java > d221112 > > ambari-server/src/main/java/org/apache/ambari/server/topology/TopologyManager.java > 5a0aca0 > > Diff: https://reviews.apache.org/r/46496/diff/ > > > Testing > --- > > Manual testign with a 5 node cluster using Blueprints. > > Unit tests: > Results : > > Tests run: 3561, Failures: 0, Errors: 0, Skipped: 36 > > > Thanks, > > Sebastian Toader > >
Re: Review Request 46508: AMBARI-15892 : Incorrect (Negative) values are shown for memory metrics (Patch 2)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46508/#review129945 --- Ship it! Ship It! - Sid Wagle On April 21, 2016, 7:10 p.m., Aravindan Vijayan wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/46508/ > --- > > (Updated April 21, 2016, 7:10 p.m.) > > > Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle. > > > Bugs: AMBARI-15892 > https://issues.apache.org/jira/browse/AMBARI-15892 > > > Repository: ambari > > > Description > --- > > Remove double subtraction of mem_cached in Yarn Cluster memory computed > metric widget. > > Change HBase Reads and Writes widget description from Count to Rate. > > Added changes that were made through AMBARI-15026 to the Upgrade path. This > makes sure hadoop daemons reported data at 10 sec intervals instead of 1 > minute interval. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog222.java > c999d52 > > ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/widgets.json > ede9f49 > > ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/YARN_widgets.json > 442d9a2 > ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/widgets.json > d8f4e6a > > ambari-server/src/main/resources/stacks/HDP/2.3/services/YARN/YARN_widgets.json > c2acf1f > > ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog222Test.java > f543f18 > > Diff: https://reviews.apache.org/r/46508/diff/ > > > Testing > --- > > Unit tests passed. > > Manually tested. > > > Thanks, > > Aravindan Vijayan > >
Re: Review Request 46602: Delete Service: Deleting Hive fails with 500 error
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46602/#review130292 --- Ship it! Ship It! - Sid Wagle On April 24, 2016, 4:40 a.m., Sumit Mohanty wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/46602/ > --- > > (Updated April 24, 2016, 4:40 a.m.) > > > Review request for Ambari, Andrew Onischuk, Jaimin Jetly, Swapan Shridhar, > and Sid Wagle. > > > Bugs: AMBARI-16080 > https://issues.apache.org/jira/browse/AMBARI-16080 > > > Repository: ambari > > > Description > --- > > The failure is due to components in STARTED state. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/state/ServiceComponentImpl.java > 93e7222 > > ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/mysql_service.py > bba5ce1 > > ambari-server/src/test/java/org/apache/ambari/server/state/ServiceComponentTest.java > 0adac80 > > Diff: https://reviews.apache.org/r/46602/diff/ > > > Testing > --- > > New unit tests and ran related ones. > > > Thanks, > > Sumit Mohanty > >
Re: Review Request 46697: export_ams_script should save configs into the files
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46697/#review130626 --- Ship it! Ship It! - Sid Wagle On April 26, 2016, 1:51 p.m., Andrew Onischuk wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/46697/ > --- > > (Updated April 26, 2016, 1:51 p.m.) > > > Review request for Ambari and Vitalyi Brodetskyi. > > > Bugs: AMBARI-16120 > https://issues.apache.org/jira/browse/AMBARI-16120 > > > Repository: ambari > > > Description > --- > > > Diffs > - > > ambari-server/src/main/resources/scripts/export_ams_metrics.py e9ef307 > > Diff: https://reviews.apache.org/r/46697/diff/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Andrew Onischuk > >
Re: Review Request 46714: Prevent Views From Causing a Loss of Service For Ambari
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46714/#review130710
---
Ship it!
Ship It!
- Sid Wagle
On April 26, 2016, 10:03 p.m., Jonathan Hurley wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46714/
> ---
>
> (Updated April 26, 2016, 10:03 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Jayush Luniya, Nate Cole, and
> Sid Wagle.
>
>
> Bugs: AMBARI-16131
> https://issues.apache.org/jira/browse/AMBARI-16131
>
>
> Repository: ambari
>
>
> Description
> ---
>
> The underlying problem is that views are accessed off of the REST endpoint
> ({{/api/v1/views}}). This means that the Ambari REST API connector is going
> to handle the request from its own threadpool. There is no way to configure
> Jetty to use a different threadpool for the same connector. As a result, if a
> request to load a view holds the Jetty thread hostage, eventually we will see
> thread starvation and loss of service.
>
> An example of this situation is a view which makes an innocent request to a
> remote resource. If the view's request has a timeout of 60 seconds, then the
> Jetty thread is going to be held for that amount of time. With concurrent
> users and multiple instances of that view deployed, the Jetty threadpool can
> becomes exhausted quickly.
>
> Although there are more graceful ways of handling this situation, they mostly
> involve substantial re-architecture and design:
> - The use of a new connector and threadpool would require binding to another
> port for view requests. This will cause problems with "local" views and their
> assumption that if they run on the Ambari server they can share the same
> session.
> - The use of a
> [Continuation|https://wiki.eclipse.org/Jetty/Feature/Continuations] in Jetty
> which can suspend the incoming request. We would need the ability for views
> to signal that they have completed their work in order to proceed with the
> suspended request.
>
> A quicker and far less invasive fix would be to create a filter which
> intercepts requests for views. It will determine how many executing view
> requests exist and decide if it will allow the new request through. For
> example, if configured to allow a maximum of 10 concurrent view requests,
> then the 11th request would be denied with an {{HTTP 503 - Service
> Unavailable}}. Although the thread is temporarily used while the filter is
> processing, it's quickly returned to the Jetty pool when it's determined
> there are too many other running view requests.
>
>
> Diffs
> -
>
>
> ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
> 5ff6a74
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
> dc53172
>
> ambari-server/src/main/java/org/apache/ambari/server/utils/VersionUtils.java
> b07f7da
> ambari-server/src/main/java/org/apache/ambari/server/view/ViewRegistry.java
> d23fcad
>
> ambari-server/src/main/java/org/apache/ambari/server/view/ViewThrottleFilter.java
> PRE-CREATION
>
> ambari-server/src/test/java/org/apache/ambari/server/view/ViewThrottleFilterTest.java
> PRE-CREATION
>
> Diff: https://reviews.apache.org/r/46714/diff/
>
>
> Testing
> ---
>
> I modified an existing view to cause it to be very naughty. It held onto the
> request for 10 seconds. Under normal Ambari operation, this caused two
> problems:
> - The view partially rendered and needed to wait
> - Ambari's API REST thread was held hostage
>
> I turned down the number of available threads to only a handful and then had
> several browsers open. This duplicated the loss of service until the view
> returned control.
>
> With the patch in place, Ambari was available and requests to the views
> beyond the configured limit would return a 503.
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Review Request 46751: Ambari metrics API call should allow for early failure
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46751/ --- Review request for Ambari, Aravindan Vijayan, Dmytro Sen, Jonathan Hurley, and Sumit Mohanty. Bugs: AMBARI-16143 https://issues.apache.org/jira/browse/AMBARI-16143 Repository: ambari Description --- In case of a SocketTimeoutException skip populating the resource with metrics until the next retry. Diffs - ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/AMSPropertyProvider.java 939ee7f ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/AMSReportPropertyProvider.java 8b60c50 ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/MetricsRequestHelper.java ca20e54 ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineMetricCache.java 9e343e3 ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineMetricCacheEntryFactory.java e8a2aef ambari-server/src/test/java/org/apache/ambari/server/controller/metrics/timeline/AMSPropertyProviderTest.java ce865cf Diff: https://reviews.apache.org/r/46751/diff/ Testing --- Added a unit test. Manually verified the patch works on a cluster. Thanks, Sid Wagle
Re: Review Request 46703: Reduce error logs on the Sink path if AMS is down
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46703/#review130881 --- Ship it! Ship It! - Sid Wagle On April 26, 2016, 3:13 p.m., Dmytro Sen wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/46703/ > --- > > (Updated April 26, 2016, 3:13 p.m.) > > > Review request for Ambari, Andrew Onischuk, Aravindan Vijayan, and Sid Wagle. > > > Bugs: AMBARI-16123 > https://issues.apache.org/jira/browse/AMBARI-16123 > > > Repository: ambari > > > Description > --- > > We should do something similar to AMSPropertyProvider where we print the > message every 1000 attempts per thread. Note: Only first message should > always be printed. > > > Diffs > - > > > ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/AbstractTimelineMetricsSink.java > 5a532c5 > > ambari-metrics/ambari-metrics-common/src/test/java/org/apache/hadoop/metrics2/sink/timeline/cache/HandleConnectExceptionTest.java > 4c1a2cb > > Diff: https://reviews.apache.org/r/46703/diff/ > > > Testing > --- > > Unit tests passed > > > Thanks, > > Dmytro Sen > >
Re: Review Request 46751: Ambari metrics API call should allow for early failure
> On April 28, 2016, 4:21 a.m., Sumit Mohanty wrote: > > ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/AMSPropertyProvider.java, > > line 209 > > <https://reviews.apache.org/r/46751/diff/1/?file=1364000#file1364000line209> > > > > should the change be conditional - for FS that can provide high > > throughput we have not had the need for these changes. IOW, what are the > > downside of these changes? Not really necessary since this used to be the behavior prior to 2.1.2 / cache introduction. We want to fail fast to avoid compounding the timeouts by making multiple AMS calls for a single Ambari request. > On April 28, 2016, 4:21 a.m., Sumit Mohanty wrote: > > ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineMetricCache.java, > > line 70 > > <https://reviews.apache.org/r/46751/diff/1/?file=1364003#file1364003line70> > > > > How long before timeout is hit? Timeouts are configurable. Cache miss = 10 seconds, Cache update = 10 (default) The update can be tuned to a lower value for a slow performing metrics system. > On April 28, 2016, 4:21 a.m., Sumit Mohanty wrote: > > ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineMetricCacheEntryFactory.java, > > line 92 > > <https://reviews.apache.org/r/46751/diff/1/?file=1364004#file1364004line92> > > > > Typically how long is the delay after which it will throw the exception. 10 seconds. - Sid --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46751/#review130880 --- On April 27, 2016, 9:47 p.m., Sid Wagle wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/46751/ > --- > > (Updated April 27, 2016, 9:47 p.m.) > > > Review request for Ambari, Aravindan Vijayan, Dmytro Sen, Jonathan Hurley, > and Sumit Mohanty. > > > Bugs: AMBARI-16143 > https://issues.apache.org/jira/browse/AMBARI-16143 > > > Repository: ambari > > > Description > --- > > In case of a SocketTimeoutException skip populating the resource with metrics > until the next retry. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/AMSPropertyProvider.java > 939ee7f > > ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/AMSReportPropertyProvider.java > 8b60c50 > > ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/MetricsRequestHelper.java > ca20e54 > > ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineMetricCache.java > 9e343e3 > > ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineMetricCacheEntryFactory.java > e8a2aef > > ambari-server/src/test/java/org/apache/ambari/server/controller/metrics/timeline/AMSPropertyProviderTest.java > ce865cf > > Diff: https://reviews.apache.org/r/46751/diff/ > > > Testing > --- > > Added a unit test. Manually verified the patch works on a cluster. > > > Thanks, > > Sid Wagle > >
Re: Review Request 46851: Express Ugrade stuck on getting status on 1600 node cluster
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46851/#review131229 --- Ship it! We did add a feature recently to allow blueprint based deploy with 0 hosts added to the cluster and thereby it might be possible to have no requests created. Not sure if it is relevant to this change though. - Sid Wagle On April 29, 2016, 11:08 p.m., Nate Cole wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/46851/ > --- > > (Updated April 29, 2016, 11:08 p.m.) > > > Review request for Ambari, Jonathan Hurley and Sid Wagle. > > > Bugs: AMBARI-16185 > https://issues.apache.org/jira/browse/AMBARI-16185 > > > Repository: ambari > > > Description > --- > > Calls to the StageResourceProvider are unnecessarily loading all stages and > tasks for Blueprint creation request. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StageResourceProvider.java > d3cb96c > > ambari-server/src/main/java/org/apache/ambari/server/topology/TopologyManager.java > b5d8d5e > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/StageResourceProviderTest.java > 2c187a9 > > Diff: https://reviews.apache.org/r/46851/diff/ > > > Testing > --- > > Manual. Automated pending. > > > Thanks, > > Nate Cole > >
Re: Review Request 46898: Provide Ability To Pass JPA / EclipseLink Properties to the DataSource
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46898/#review131440
---
Ship it!
Awesome !! We definetly needed this.
- Sid Wagle
On May 2, 2016, 6:17 p.m., Jonathan Hurley wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46898/
> ---
>
> (Updated May 2, 2016, 6:17 p.m.)
>
>
> Review request for Ambari, Nate Cole and Sid Wagle.
>
>
> Bugs: AMBARI-16207
> https://issues.apache.org/jira/browse/AMBARI-16207
>
>
> Repository: ambari
>
>
> Description
> ---
>
> There is currently no way to pass JPA or EclipseLink specific
> connection/DataSource properties in from {{ambari.properties}}. Although
> there exists {{Configuration.getDatabaseCustomProperties}}, these are
> actually _driver specific_ properties and not properties for EclipseLink.
>
> For example, if I wanted to pass in to the JDBC Driver {{foo=bar}}, then I
> could set
> {{server.jdbc.properties.foo=bar}} and this will get translated into
> {{eclipselink.jdbc.property.foo=bar}}.
>
> However, if I wanted to set any of the EclipseLink or JPA specific DataSource
> properties (see
> http://www.eclipse.org/eclipselink/api/2.6/org/eclipse/persistence/config/PersistenceUnitProperties.html)
> I would not be able to.
>
> Proposal is to add something similar to the custom driver properties:
> ```
> server.persistence.properties.eclipselink.jdbc.batch-writing.size=25
> ```
>
> Which could get translated into
> ```
> properties.put("eclipselink.jdbc.batch-writing.size", 25)
> ```
>
>
> Diffs
> -
>
>
> ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
> 0afae97
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java
> 3506031
>
> ambari-server/src/test/java/org/apache/ambari/server/configuration/ConfigurationTest.java
> 99ec786
>
> Diff: https://reviews.apache.org/r/46898/diff/
>
>
> Testing
> ---
>
> mvn clean test
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 47014: Blueprint processor should create ConfigGroup even with only one host registered
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47014/#review131851 --- Don't the subsequent CG calls need to do an update vs create? Unless we are always creating CG with configs and no hosts and then update request with hosts. - Sid Wagle On May 5, 2016, 3:55 p.m., Sebastian Toader wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/47014/ > --- > > (Updated May 5, 2016, 3:55 p.m.) > > > Review request for Ambari, Daniel Gergely, Laszlo Puskas, Sandor Magyari, > Srimanth Gunturi, and Sid Wagle. > > > Bugs: AMBARI-16270 > https://issues.apache.org/jira/browse/AMBARI-16270 > > > Repository: ambari > > > Description > --- > > When the first host for a host group registers with the cluster and is > assigned to host group Ambari checks if there is any config group to be > created for that host group. If it has to than create a config group and > assigning to it all hosts that are expected for the parent host group. > However it not checks is all of the expected hosts of the host group have > already been associated with the cluster thus the config group creation fails > as config groups can have only hosts added that already are associated with > the cluster. > > This has been changed to add only those hosts to config group that already > have been associated with the cluster and update the created config group > with the remaing hosts as these become available and register with the > cluster. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/topology/AmbariContext.java > cf1a6ac > > ambari-server/src/test/java/org/apache/ambari/server/topology/AmbariContextTest.java > 1613d11 > > Diff: https://reviews.apache.org/r/47014/diff/ > > > Testing > --- > > Manual testing done. > > Unit test results: > Results : > > Tests run: 3568, Failures: 0, Errors: 0, Skipped: 36 > > > Thanks, > > Sebastian Toader > >
Re: Review Request 47014: Blueprint processor should create ConfigGroup even with only one host registered
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47014/#review131873 --- Ship it! Ship It! - Sid Wagle On May 5, 2016, 3:55 p.m., Sebastian Toader wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/47014/ > --- > > (Updated May 5, 2016, 3:55 p.m.) > > > Review request for Ambari, Daniel Gergely, Laszlo Puskas, Sandor Magyari, > Srimanth Gunturi, and Sid Wagle. > > > Bugs: AMBARI-16270 > https://issues.apache.org/jira/browse/AMBARI-16270 > > > Repository: ambari > > > Description > --- > > When the first host for a host group registers with the cluster and is > assigned to host group Ambari checks if there is any config group to be > created for that host group. If it has to than create a config group and > assigning to it all hosts that are expected for the parent host group. > However it not checks is all of the expected hosts of the host group have > already been associated with the cluster thus the config group creation fails > as config groups can have only hosts added that already are associated with > the cluster. > > This has been changed to add only those hosts to config group that already > have been associated with the cluster and update the created config group > with the remaing hosts as these become available and register with the > cluster. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/topology/AmbariContext.java > cf1a6ac > > ambari-server/src/test/java/org/apache/ambari/server/topology/AmbariContextTest.java > 1613d11 > > Diff: https://reviews.apache.org/r/47014/diff/ > > > Testing > --- > > Manual testing done. > > Unit test results: > Results : > > Tests run: 3568, Failures: 0, Errors: 0, Skipped: 36 > > > Thanks, > > Sebastian Toader > >
Re: Review Request 47142: AMBARI-16412 : Support TopN queries in AMS
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47142/#review132356 --- Fix it, then Ship it! ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/TopNConfig.java (line 21) <https://reviews.apache.org/r/47142/#comment196529> Missing JSON annotations. ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java (line 218) <https://reviews.apache.org/r/47142/#comment196530> Lets make this configurable, no need to add config to the xml. ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java (line 308) <https://reviews.apache.org/r/47142/#comment196535> We should cleanup the interface and remove this sigular function call unless it is documented as supported API. ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricStore.java (line 53) <https://reviews.apache.org/r/47142/#comment196540> formatting ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/PhoenixTransactSQL.java (line 472) <https://reviews.apache.org/r/47142/#comment196541> Can we do refactor this? Builder would be great. ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/PhoenixTransactSQL.java (line 872) <https://reviews.apache.org/r/47142/#comment196542> Rename to getTargetTableUsingPrecision ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/TopNCondition.java (line 146) <https://reviews.apache.org/r/47142/#comment196543> Add javadoc for method - Sid Wagle On May 9, 2016, 11:50 p.m., Aravindan Vijayan wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/47142/ > --- > > (Updated May 9, 2016, 11:50 p.m.) > > > Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle. > > > Bugs: AMBARI-16412 > https://issues.apache.org/jira/browse/AMBARI-16412 > > > Repository: ambari > > > Description > --- > > Adding TopN support to AMS API > > Following are supported. > > 1. Top "N" hosts when 1 metric is requested for "H" hosts. > 2. Top "N" metrics when "M" metrics (or a pattern) are requested for 0 or 1 > hosts. > 3. When more than 20 hosts are requested, AMS defaults to Top N hosts > condition with N=20. > > > Diffs > - > > > ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/TopNConfig.java > PRE-CREATION > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java > 89c67d1 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java > a0ecbcc > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricStore.java > ded64e3 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricStoreWatcher.java > 632df3f > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/ConditionBuilder.java > PRE-CREATION > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/DefaultCondition.java > 99a6125 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/PhoenixTransactSQL.java > 6ee0006 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/TopNCondition.java > PRE-CREATION > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoo
Re: Review Request 47142: AMBARI-16412 : Support TopN queries in AMS
> On May 10, 2016, 12:27 a.m., Sid Wagle wrote: > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java, > > line 308 > > <https://reviews.apache.org/r/47142/diff/1/?file=1377200#file1377200line308> > > > > We should cleanup the interface and remove this sigular function call > > unless it is documented as supported API. > > Aravindan Vijayan wrote: > Used by some unit tests. The unit tests can be ignored as long this API is not being used in order to avoid maintenance headache. - Sid --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47142/#review132356 --- On May 9, 2016, 11:50 p.m., Aravindan Vijayan wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/47142/ > --- > > (Updated May 9, 2016, 11:50 p.m.) > > > Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle. > > > Bugs: AMBARI-16412 > https://issues.apache.org/jira/browse/AMBARI-16412 > > > Repository: ambari > > > Description > --- > > Adding TopN support to AMS API > > Following are supported. > > 1. Top "N" hosts when 1 metric is requested for "H" hosts. > 2. Top "N" metrics when "M" metrics (or a pattern) are requested for 0 or 1 > hosts. > 3. When more than 20 hosts are requested, AMS defaults to Top N hosts > condition with N=20. > > > Diffs > - > > > ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/TopNConfig.java > PRE-CREATION > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java > 89c67d1 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java > a0ecbcc > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricStore.java > ded64e3 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricStoreWatcher.java > 632df3f > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/ConditionBuilder.java > PRE-CREATION > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/DefaultCondition.java > 99a6125 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/PhoenixTransactSQL.java > 6ee0006 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/TopNCondition.java > PRE-CREATION > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/webapp/TimelineWebServices.java > 4cfc415 > > ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TestPhoenixTransactSQL.java > 9c6617c > > ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TestTimelineMetricStore.java > 16bbf0e > > ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricStoreWatcherTest.java > 277a98c > > Diff: https://reviews.apache.org/r/47142/diff/ > > > Testing > --- > > Manually tested. > Unit tests pass. > > > Thanks, > > Aravindan Vijayan > >
Re: Review Request 47147: AMBARI-16413: Ambari server does not start because of permission issues on symbolic links
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47147/#review132490 --- Ship it! Ship It! - Sid Wagle On May 10, 2016, 12:41 a.m., Nahappan Somasundaram wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/47147/ > --- > > (Updated May 10, 2016, 12:41 a.m.) > > > Review request for Ambari, Andrew Onischuk, Mahadev Konar, and Sumit Mohanty. > > > Bugs: AMBARI-16413 > https://issues.apache.org/jira/browse/AMBARI-16413 > > > Repository: ambari > > > Description > --- > > AMBARI-16413: Ambari server does not start because of permission issues on > symbolic links > > ** Issue ** > > When setting up ambari-server with a non-root user, ownership change on > various log files and other files do not work when those files are in a > symbolic linked folder. > > ** Fix ** > When doing a chown -R , specify the -L option to traverse every > symbolic link. > > > Diffs > - > > ambari-common/src/main/python/ambari_commons/os_linux.py a64f99a > > Diff: https://reviews.apache.org/r/47147/diff/ > > > Testing > --- > > ** 1. mvn test -DskipSurefireTests ** > > -- > Ran 261 tests in 6.567s > > OK > -- > Total run:1014 > Total errors:0 > Total failures:0 > OK > INFO: AMBARI_SERVER_LIB is not set, using default /usr/lib/ambari-server > INFO: Return code from stack upgrade command, retcode = 0 > StackAdvisor implementation for stack HDP1, version 2.0.6 was not found > Returning DefaultStackAdvisor implementation > StackAdvisor implementation for stack XYZ, version 1.0.0 was loaded > StackAdvisor implementation for stack XYZ, version 1.0.1 was loaded > Returning XYZ101StackAdvisor implementation > [INFO] > > [INFO] BUILD SUCCESS > [INFO] > > [INFO] Total time: 1:06.431s > [INFO] Finished at: Mon May 09 17:35:47 PDT 2016 > [INFO] Final Memory: 61M/984M > [INFO] > > > ** 2. Manual testing ** > Set up a VM and copy the changed os_linux.py script fromt the build to the > VM. Run the following commands: > rm -rf /var/log/ambari-server > mkdir /tmp/log/ambari-server > cd /var/log > ln -s ambari-server /tmp/log/ambari-server > > Run ambari-server setup and use a non-root user. Verify that the permissions > on /var/log/ambari-server and it's contents now have the non-root user as the > owner. > > Run ambari-server start and verify that it was successful. > > > Thanks, > > Nahappan Somasundaram > >
Re: Review Request 47142: AMBARI-16412 : Support TopN queries in AMS
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47142/#review132493 --- ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/PhoenixTransactSQL.java (line 301) <https://reviews.apache.org/r/47142/#comment196716> Is the native timerange hint applicable to both SELECT queries? - Sid Wagle On May 10, 2016, 5:34 p.m., Aravindan Vijayan wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/47142/ > --- > > (Updated May 10, 2016, 5:34 p.m.) > > > Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle. > > > Bugs: AMBARI-16412 > https://issues.apache.org/jira/browse/AMBARI-16412 > > > Repository: ambari > > > Description > --- > > Adding TopN support to AMS API > > Following are supported. > > 1. Top "N" hosts when 1 metric is requested for "H" hosts. > 2. Top "N" metrics when "M" metrics (or a pattern) are requested for 0 or 1 > hosts. > 3. When more than 20 hosts are requested, AMS defaults to Top N hosts > condition with N=20. > > > Diffs > - > > > ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/TopNConfig.java > PRE-CREATION > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java > 89c67d1 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java > a0ecbcc > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricConfiguration.java > cf30e24 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricStore.java > ded64e3 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricStoreWatcher.java > 632df3f > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/ConditionBuilder.java > PRE-CREATION > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/DefaultCondition.java > 99a6125 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/PhoenixTransactSQL.java > 6ee0006 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/TopNCondition.java > PRE-CREATION > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/webapp/TimelineWebServices.java > 4cfc415 > > ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TestPhoenixTransactSQL.java > 9c6617c > > ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TestTimelineMetricStore.java > 16bbf0e > > ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricStoreWatcherTest.java > 277a98c > > Diff: https://reviews.apache.org/r/47142/diff/ > > > Testing > --- > > Manually tested. > Unit tests pass. > > > Thanks, > > Aravindan Vijayan > >
Re: Review Request 47142: AMBARI-16412 : Support TopN queries in AMS
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47142/#review132514 --- Ship it! Ship It! - Sid Wagle On May 10, 2016, 6:41 p.m., Aravindan Vijayan wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/47142/ > --- > > (Updated May 10, 2016, 6:41 p.m.) > > > Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle. > > > Bugs: AMBARI-16412 > https://issues.apache.org/jira/browse/AMBARI-16412 > > > Repository: ambari > > > Description > --- > > Adding TopN support to AMS API > > Following are supported. > > 1. Top "N" hosts when 1 metric is requested for "H" hosts. > 2. Top "N" metrics when "M" metrics (or a pattern) are requested for 0 or 1 > hosts. > 3. When more than 20 hosts are requested, AMS defaults to Top N hosts > condition with N=20. > > > Diffs > - > > > ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/TopNConfig.java > PRE-CREATION > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java > 89c67d1 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java > a0ecbcc > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricConfiguration.java > cf30e24 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricStore.java > ded64e3 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricStoreWatcher.java > 632df3f > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/ConditionBuilder.java > PRE-CREATION > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/DefaultCondition.java > 99a6125 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/PhoenixTransactSQL.java > 6ee0006 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/TopNCondition.java > PRE-CREATION > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/webapp/TimelineWebServices.java > 4cfc415 > > ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TestPhoenixTransactSQL.java > 9c6617c > > ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TestTimelineMetricStore.java > 16bbf0e > > ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricStoreWatcherTest.java > 277a98c > > Diff: https://reviews.apache.org/r/47142/diff/ > > > Testing > --- > > Manually tested. > Unit tests pass. > > > Thanks, > > Aravindan Vijayan > >
Re: Review Request 47235: Takeover script on Paypal configs results in conflict message that look wrong
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47235/#review132667 --- Ship it! Ship It! - Sid Wagle On May 11, 2016, 1:54 p.m., Andrew Onischuk wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/47235/ > --- > > (Updated May 11, 2016, 1:54 p.m.) > > > Review request for Ambari, Srimanth Gunturi and Sid Wagle. > > > Bugs: AMBARI-16451 > https://issues.apache.org/jira/browse/AMBARI-16451 > > > Repository: ambari > > > Description > --- > > Attaching the zip file with configs and conflicts. > > * The conflicting configs seem to have a lot of duplicate info > mapred-site :: dfs.datanode.du.reserved > > * Unable to handle empty config files: > Had to hand edit and add try / catch at : > > > yaml.load(file).iteritems() > > > parsed_configurations_from_path, parsed_properties_attributes = > parser.read_data_to_map(path) > > > > > Traceback (most recent call last): > File "./takeover_config_merge.py", line 393, in > sys.exit(main()) > File "./takeover_config_merge.py", line 389, in main > return configMerge.perform_merge() > File "./takeover_config_merge.py", line 279, in perform_merge > parsed_configurations_from_path, parsed_properties_attributes = > parser.read_data_to_map(path) > File "./takeover_config_merge.py", line 86, in read_data_to_map > for name, value in yaml.load(file).iteritems(): > AttributeError: 'NoneType' object has no attribute 'iteritems' > > > _Instruction_ > > > > > ./takeover_config_merge.py -o blueprint -i files > > > Diffs > - > > ambari-server/src/main/resources/scripts/takeover_config_merge.py 1432e4b > > Diff: https://reviews.apache.org/r/47235/diff/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Andrew Onischuk > >
Re: Review Request 47244: AMBARI-16440 : Flush metrics to collector if metric system is stopped gracefully in the Sink daemon
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47244/#review132693 --- ambari-metrics/ambari-metrics-hadoop-sink/src/main/java/org/apache/hadoop/metrics2/sink/timeline/HadoopTimelineMetricsSink.java (line 395) <https://reviews.apache.org/r/47244/#comment196961> Why not keep the same thread and just submit task to it? - Sid Wagle On May 11, 2016, 5 p.m., Aravindan Vijayan wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/47244/ > --- > > (Updated May 11, 2016, 5 p.m.) > > > Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle. > > > Bugs: AMBARI-16440 > https://issues.apache.org/jira/browse/AMBARI-16440 > > > Repository: ambari > > > Description > --- > > HBase Regionserver daemons shuts down and restarts the Metric System for > every 5 minutes to make sure there is no growth of metrics. This causes AMS > Hadoop sink to lose metrics intermittently, which was buffered during that > time. This leads to holes in data and negative rates in graphs. > > > Diffs > - > > > ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/cache/TimelineMetricsCache.java > 3316a54 > > ambari-metrics/ambari-metrics-hadoop-sink/src/main/java/org/apache/hadoop/metrics2/sink/timeline/HadoopTimelineMetricsSink.java > 35b9459 > > Diff: https://reviews.apache.org/r/47244/diff/ > > > Testing > --- > > Manually tested. > > Unit tests pending. > > > Thanks, > > Aravindan Vijayan > >
Re: Review Request 46661: Put HBase master UI into readonly mode for secure deployment for HDP 2.5
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46661/#review132801 --- Ship it! Ship It! - Sid Wagle On May 11, 2016, 11:31 p.m., Ajit Kumar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/46661/ > --- > > (Updated May 11, 2016, 11:31 p.m.) > > > Review request for Ambari, Robert Levas, Sumit Mohanty, and Sid Wagle. > > > Bugs: AMBARI-16164 > https://issues.apache.org/jira/browse/AMBARI-16164 > > > Repository: ambari > > > Description > --- > > Currently in secure deployment, user can request compaction / splitting > through hbase master UI. > This potentially exposes vulnerability to various attacks. > There is config parameter, hbase.master.ui.readonly, with default value of > false. > In secure deployment, Master UI should be put to readonly mode (setting the > above parameter to true). > Admin can always request compaction / splitting through hbase shell. > > > Diffs > - > > ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py > 3d689d7dd4f9c5bd98d5d4e0152775fd7ea012a2 > > Diff: https://reviews.apache.org/r/46661/diff/ > > > Testing > --- > > Manual testing, sceenshot attached (after kerberizing cluster). > > > File Attachments > > > hbase-site.png > > https://reviews.apache.org/media/uploaded/files/2016/05/11/75e8d2ae-99a4-4e8d-8a69-52cd974734fb__hbase-site.png > > > Thanks, > > Ajit Kumar > >
Re: Review Request 47369: AMBARI-16666 : AMS Service check fails because collector takes too long to start up.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47369/#review133280 --- Ship it! Ship It! - Sid Wagle On May 13, 2016, 10:17 p.m., Aravindan Vijayan wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/47369/ > --- > > (Updated May 13, 2016, 10:17 p.m.) > > > Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle. > > > Bugs: AMBARI-1 > https://issues.apache.org/jira/browse/AMBARI-1 > > > Repository: ambari > > > Description > --- > > Problem > If etc/hosts is not properly configured, AMS embedded mode startup takes too > long because of connection timeout to localhost zk quorum. > > Fix > Change embedded mode zk quorum to hostname from localhost. > > > Diffs > - > > > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py > 32b8fdb > > Diff: https://reviews.apache.org/r/47369/diff/ > > > Testing > --- > > Manaully tested embedded mode on secure and unsecure cluster. > Python unit tests pass. > > > Thanks, > > Aravindan Vijayan > >
Re: Review Request 47244: AMBARI-16440 : Flush metrics to collector if metric system is stopped gracefully in the Sink daemon
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47244/#review133281 --- Ship it! Ship It! - Sid Wagle On May 13, 2016, 8:23 p.m., Aravindan Vijayan wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/47244/ > --- > > (Updated May 13, 2016, 8:23 p.m.) > > > Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle. > > > Bugs: AMBARI-16440 > https://issues.apache.org/jira/browse/AMBARI-16440 > > > Repository: ambari > > > Description > --- > > HBase Regionserver daemons shuts down and restarts the Metric System for > every 5 minutes to make sure there is no growth of metrics. This causes AMS > Hadoop sink to lose metrics intermittently, which was buffered during that > time. This leads to holes in data and negative rates in graphs. > > > Diffs > - > > > ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/cache/TimelineMetricsCache.java > 3316a54 > > ambari-metrics/ambari-metrics-hadoop-sink/src/main/java/org/apache/hadoop/metrics2/sink/timeline/HadoopTimelineMetricsSink.java > 35b9459 > > Diff: https://reviews.apache.org/r/47244/diff/ > > > Testing > --- > > Manually tested. > > Unit tests passed. > > > Thanks, > > Aravindan Vijayan > >
Review Request 47420: Support grafana dashboards to be defined based on stack
/0.1.0/package/files/grafana-dashboards/grafana-hive-hiverserver2.json b77660d ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/files/grafana-dashboards/grafana-hive-home.json 9cc50a6 ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/files/grafana-dashboards/grafana-llapdaemon-daemons.json 3b9184e ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/files/grafana-dashboards/grafana-llapdaemon-heatmaps.json 944b37e ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/files/grafana-dashboards/grafana-llapdaemon-overview.json 63f5a99 ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/files/grafana-dashboards/grafana-system-home.json e967fcb ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/files/grafana-dashboards/grafana-system-servers.json 3846288 ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/files/grafana-dashboards/grafana-yarn-applications.json 484a965 ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/files/grafana-dashboards/grafana-yarn-home.json 5b812c0 ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/files/grafana-dashboards/grafana-yarn-jobhistoryserver.json 4e8bc7c ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/files/grafana-dashboards/grafana-yarn-nodemanagers.json bfe8829 ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/files/grafana-dashboards/grafana-yarn-queues.json 860ee2f ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/files/grafana-dashboards/grafana-yarn-resourcemanagers.json e9a3d30 ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/files/grafana-dashboards/grafana-yarn-timelineserver.json fba32bd ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py b3ab0be Diff: https://reviews.apache.org/r/47420/diff/ Testing --- -- Ran 261 tests in 6.699s OK -- Total run:1016 Total errors:0 Total failures:0 OK Thanks, Sid Wagle
Re: Review Request 43126: Ambari Upgrade should clean older RCA Database tables
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/43126/#review133450 --- What about RCA tables not a part of Ambari DB ? - Sid Wagle On May 16, 2016, 9:28 p.m., Ajit Kumar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/43126/ > --- > > (Updated May 16, 2016, 9:28 p.m.) > > > Review request for Ambari, Jayush Luniya, Sumit Mohanty, and Sid Wagle. > > > Bugs: AMBARI-16692 > https://issues.apache.org/jira/browse/AMBARI-16692 > > > Repository: ambari > > > Description > --- > > RCA Database tables is not used anymore. This older, un-necessary data causes > many Ambari DB queries to return more slowly than necessary. If the database > gets large enough, it can have significant impact on the way ambari-server > functions. > The Ambari Server upgrade process should detect the presence of these older, > deprecated tables, and automate the process of cleaning up this data. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java > 3547ad3 > > ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog240Test.java > f36e640 > > Diff: https://reviews.apache.org/r/43126/diff/ > > > Testing > --- > > Unit test > > > Thanks, > > Ajit Kumar > >
Re: Review Request 47420: Support grafana dashboards to be defined based on stack
/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/files/grafana-dashboards/grafana-hive-hiverserver2.json b77660d ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/files/grafana-dashboards/grafana-hive-home.json 9cc50a6 ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/files/grafana-dashboards/grafana-llapdaemon-daemons.json 3b9184e ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/files/grafana-dashboards/grafana-llapdaemon-heatmaps.json 944b37e ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/files/grafana-dashboards/grafana-llapdaemon-overview.json 63f5a99 ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/files/grafana-dashboards/grafana-system-home.json e967fcb ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/files/grafana-dashboards/grafana-system-servers.json 3846288 ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/files/grafana-dashboards/grafana-yarn-applications.json 484a965 ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/files/grafana-dashboards/grafana-yarn-home.json 5b812c0 ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/files/grafana-dashboards/grafana-yarn-jobhistoryserver.json 4e8bc7c ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/files/grafana-dashboards/grafana-yarn-nodemanagers.json bfe8829 ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/files/grafana-dashboards/grafana-yarn-queues.json 860ee2f ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/files/grafana-dashboards/grafana-yarn-resourcemanagers.json e9a3d30 ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/files/grafana-dashboards/grafana-yarn-timelineserver.json fba32bd ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_grafana_util.py 15c96dd ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py b3ab0be Diff: https://reviews.apache.org/r/47420/diff/ Testing --- -- Ran 261 tests in 6.699s OK -- Total run:1016 Total errors:0 Total failures:0 OK Thanks, Sid Wagle
Re: Review Request 43126: Ambari Upgrade should clean older RCA Database tables
> On May 16, 2016, 9:47 p.m., Sid Wagle wrote: > > What about RCA tables not a part of Ambari DB ? > > Ajit Kumar wrote: > Only postgres-embedded-create.sql has ambarirca db. Users who are facing > this issue have stand alone DB server and don't use postgres-embedded. > Ideally we should remove ambarirca db from postgres-embedded-create.sql as > well. We should it as a part of this patch IMO. The rca tables are no longer used. Prior to 1.5.1 we had separate RCA database on the same database server. We should in all likelyhood cleaning that up as well, no? ambari.properties has the hdbc URL for it. - Sid --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/43126/#review133450 --- On May 16, 2016, 9:28 p.m., Ajit Kumar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/43126/ > --- > > (Updated May 16, 2016, 9:28 p.m.) > > > Review request for Ambari, Jayush Luniya, Sumit Mohanty, and Sid Wagle. > > > Bugs: AMBARI-16692 > https://issues.apache.org/jira/browse/AMBARI-16692 > > > Repository: ambari > > > Description > --- > > RCA Database tables is not used anymore. This older, un-necessary data causes > many Ambari DB queries to return more slowly than necessary. If the database > gets large enough, it can have significant impact on the way ambari-server > functions. > The Ambari Server upgrade process should detect the presence of these older, > deprecated tables, and automate the process of cleaning up this data. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java > 3547ad3 > > ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog240Test.java > f36e640 > > Diff: https://reviews.apache.org/r/43126/diff/ > > > Testing > --- > > Unit test > > > Thanks, > > Ajit Kumar > >
Review Request 47438: HDFS Alerts: add minimum values to AMS alerts
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47438/ --- Review request for Ambari, Aravindan Vijayan, Sumit Mohanty, and Srimanth Gunturi. Bugs: AMBARI-16695 https://issues.apache.org/jira/browse/AMBARI-16695 Repository: ambari Description --- There are new HDFS alerts that watch growth rates. Some (like RPC) have "minimum" values, meaning we ignore growth until we are past a certain value (like latency in seconds). There are a few other alerts that I think also need minimums. Diffs - ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/alerts.json aedbdfe ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/alerts/alert_metrics_deviation.py 3296493 Diff: https://reviews.apache.org/r/47438/diff/ Testing --- Deployed manually to verify nothing breaks. -- Ran 261 tests in 6.682s OK -- Total run:1016 Total errors:0 Total failures:0 OK Thanks, Sid Wagle
Re: Review Request 47438: HDFS Alerts: add minimum values to AMS alerts
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47438/ --- (Updated May 17, 2016, 12:51 a.m.) Review request for Ambari, Aravindan Vijayan, Sumit Mohanty, and Srimanth Gunturi. Changes --- Comment typo. Bugs: AMBARI-16695 https://issues.apache.org/jira/browse/AMBARI-16695 Repository: ambari Description --- There are new HDFS alerts that watch growth rates. Some (like RPC) have "minimum" values, meaning we ignore growth until we are past a certain value (like latency in seconds). There are a few other alerts that I think also need minimums. Diffs (updated) - ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/alerts.json aedbdfe ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/alerts/alert_metrics_deviation.py 3296493 Diff: https://reviews.apache.org/r/47438/diff/ Testing --- Deployed manually to verify nothing breaks. -- Ran 261 tests in 6.682s OK -- Total run:1016 Total errors:0 Total failures:0 OK Thanks, Sid Wagle
Re: Review Request 47438: HDFS Alerts: add minimum values to AMS alerts
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47438/#review133478 --- Just realized I am missing UpgradeCatalog changes, will add subsequent patch. - Sid Wagle On May 17, 2016, 12:51 a.m., Sid Wagle wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/47438/ > --- > > (Updated May 17, 2016, 12:51 a.m.) > > > Review request for Ambari, Aravindan Vijayan, Sumit Mohanty, and Srimanth > Gunturi. > > > Bugs: AMBARI-16695 > https://issues.apache.org/jira/browse/AMBARI-16695 > > > Repository: ambari > > > Description > --- > > There are new HDFS alerts that watch growth rates. Some (like RPC) have > "minimum" values, meaning we ignore growth until we are past a certain value > (like latency in seconds). > > There are a few other alerts that I think also need minimums. > > > Diffs > - > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/alerts.json > aedbdfe > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/alerts/alert_metrics_deviation.py > 3296493 > > Diff: https://reviews.apache.org/r/47438/diff/ > > > Testing > --- > > Deployed manually to verify nothing breaks. > > -- > Ran 261 tests in 6.682s > > OK > -- > Total run:1016 > Total errors:0 > Total failures:0 > OK > > > Thanks, > > Sid Wagle > >
Re: Review Request 47456: Takeover config merge should handle AMS hbase configs
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47456/#review133631 --- ambari-server/src/main/resources/scripts/takeover_files_mapping.json (line 2) <https://reviews.apache.org/r/47456/#comment198178> Lets add this for ams-hbase-env, ams-hbase-log4j, - Sid Wagle On May 17, 2016, 2:16 p.m., Andrew Onischuk wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/47456/ > --- > > (Updated May 17, 2016, 2:16 p.m.) > > > Review request for Ambari, Sid Wagle and Vitalyi Brodetskyi. > > > Bugs: AMBARI-16703 > https://issues.apache.org/jira/browse/AMBARI-16703 > > > Repository: ambari > > > Description > --- > > hbase-site conflicts between cluster and AMS. > > Hard code the config path as /etc/ams-hbase/conf to be AMS HBase configs. > > > Diffs > - > > ambari-server/src/main/resources/scripts/takeover_config_merge.py f940f27 > ambari-server/src/main/resources/scripts/takeover_files_mapping.json > PRE-CREATION > > Diff: https://reviews.apache.org/r/47456/diff/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Andrew Onischuk > >
Re: Review Request 47438: HDFS Alerts: add minimum values to AMS alerts
> On May 17, 2016, 1:52 a.m., Sid Wagle wrote: > > Just realized I am missing UpgradeCatalog changes, will add subsequent > > patch. No need for upgrade steps since the Catalog deletes all standard deviation alerts which get added back on server restart. - Sid --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47438/#review133478 --- On May 17, 2016, 12:51 a.m., Sid Wagle wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/47438/ > --- > > (Updated May 17, 2016, 12:51 a.m.) > > > Review request for Ambari, Aravindan Vijayan, Sumit Mohanty, and Srimanth > Gunturi. > > > Bugs: AMBARI-16695 > https://issues.apache.org/jira/browse/AMBARI-16695 > > > Repository: ambari > > > Description > --- > > There are new HDFS alerts that watch growth rates. Some (like RPC) have > "minimum" values, meaning we ignore growth until we are past a certain value > (like latency in seconds). > > There are a few other alerts that I think also need minimums. > > > Diffs > - > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/alerts.json > aedbdfe > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/alerts/alert_metrics_deviation.py > 3296493 > > Diff: https://reviews.apache.org/r/47438/diff/ > > > Testing > --- > > Deployed manually to verify nothing breaks. > > -- > Ran 261 tests in 6.682s > > OK > -- > Total run:1016 > Total errors:0 > Total failures:0 > OK > > > Thanks, > > Sid Wagle > >
Re: Review Request 47595: Implement in ams collector batch insert operations to ams-hbase
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47595/#review134092 --- Fix it, then Ship it! ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java (line 205) <https://reviews.apache.org/r/47595/#comment198735> Lets rename to cacheComitInterval vs sending ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java (line 232) <https://reviews.apache.org/r/47595/#comment198736> again we should call this commit vs send. ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java (line 695) <https://reviews.apache.org/r/47595/#comment198737> Lets use >= instead, sicne not sure if size() is threadsafe ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricConfiguration.java (line 52) <https://reviews.apache.org/r/47595/#comment198738> Rename to evict or commit. ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml (line 557) <https://reviews.apache.org/r/47595/#comment198739> Units should be clear from name or description of property. - Sid Wagle On May 19, 2016, 2:43 p.m., Dmytro Sen wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/47595/ > --- > > (Updated May 19, 2016, 2:43 p.m.) > > > Review request for Ambari, Aravindan Vijayan and Sid Wagle. > > > Bugs: AMBARI-16766 > https://issues.apache.org/jira/browse/AMBARI-16766 > > > Repository: ambari > > > Description > --- > > Check if there is performance boost after replacing insert operations with > bulk loading. > If yes, implement and provide switch property for enabling/disabling this > feature. > > > Diffs > - > > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java > 974f951 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/MetricsCacheCommitter.java > PRE-CREATION > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java > 52ab083 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricConfiguration.java > 683e5d4 > > ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessorTest.java > 290a98a > > ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/discovery/TestMetadataManager.java > 06c71c5 > > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml > a7fd9c8 > > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/service_check.py > 8f369f7 > > Diff: https://reviews.apache.org/r/47595/diff/ > > > Testing > --- > > Unit tests passed > > > Thanks, > > Dmytro Sen > >
