Re: Review Request 64989: AMBARI-22740 : Fix integration test for HBase in branch-3.0-ams due to UUID changes
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64989/#review194921 --- Ship it! Ship It! - Sid Wagle On Jan. 5, 2018, 11:19 p.m., Aravindan Vijayan wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64989/ > --- > > (Updated Jan. 5, 2018, 11:19 p.m.) > > > Review request for Ambari, Dmytro Sen and Sid Wagle. > > > Bugs: AMBARI-22740 > https://issues.apache.org/jira/browse/AMBARI-22740 > > > Repository: ambari > > > Description > --- > > Fix integration test for HBase in branch-3.0-ams due to UUID changes > > > Diffs > - > > > ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/Precision.java > e87f06e > > ambari-metrics/ambari-metrics-timelineservice/conf/unix/ambari-metrics-collector > de764ec > ambari-metrics/ambari-metrics-timelineservice/pom.xml d3a976b > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/ambari/metrics/AMSApplicationServer.java > PRE-CREATION > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/ambari/metrics/core/loadsimulator/LoadRunner.java > PRE-CREATION > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/ambari/metrics/core/loadsimulator/MetricsLoadSimulator.java > PRE-CREATION > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/ambari/metrics/core/loadsimulator/MetricsSenderWorker.java > PRE-CREATION > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/ambari/metrics/core/loadsimulator/data/AppID.java > PRE-CREATION > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/ambari/metrics/core/loadsimulator/data/AppMetrics.java > PRE-CREATION > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/ambari/metrics/core/loadsimulator/data/ApplicationInstance.java > PRE-CREATION > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/ambari/metrics/core/loadsimulator/data/HostMetricsGenerator.java > PRE-CREATION > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/ambari/metrics/core/loadsimulator/data/Metric.java > PRE-CREATION > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/ambari/metrics/core/loadsimulator/data/MetricsGeneratorConfigurer.java > PRE-CREATION > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/ambari/metrics/core/loadsimulator/net/MetricsSender.java > PRE-CREATION > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/ambari/metrics/core/loadsimulator/net/RestMetricsSender.java > PRE-CREATION > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/ambari/metrics/core/loadsimulator/net/StdOutMetricsSender.java > PRE-CREATION > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/ambari/metrics/core/loadsimulator/net/UrlService.java > PRE-CREATION > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/ambari/metrics/core/loadsimulator/util/Json.java > PRE-CREATION > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/ambari/metrics/core/loadsimulator/util/RandomMetricsProvider.java > PRE-CREATION > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/ambari/metrics/core/loadsimulator/util/TimeStampProvider.java > PRE-CREATION > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/ambari/metrics/core/timeline/HBaseTimelineMetricsService.java > PRE-CREATION > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/ambari/metrics/core/timeline/MetricsCacheCommitterThread.java > PRE-CREATION > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/ambari/metrics/core/timeline/MetricsSystemInitializationException.java > PRE-CREATION > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/ambari/metrics/core/timeline/PhoenixHBaseAccessor.java > PRE-CREATION > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/ambari/metrics/core/timeline/TimelineMetricConfiguration.java > PRE-CREATION > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/ambari/metrics/core/timeline
Re: Review Request 64990: Add event for changes in upgrade
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64990/#review194903 --- Fix it, then Ship it! ambari-server/src/main/java/org/apache/ambari/server/events/UpgradeUpdateEvent.java Lines 38 (patched) <https://reviews.apache.org/r/64990/#comment273985> Can you add some javadoc please sa to what this event represents. - Sid Wagle On Jan. 5, 2018, 8:13 p.m., Myroslav Papirkovskyy wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64990/ > --- > > (Updated Jan. 5, 2018, 8:13 p.m.) > > > Review request for Ambari, Andrew Onischuk, Dmytro Grinenko, Eugene > Chekanskiy, and Sid Wagle. > > > Bugs: AMBARI-22742 > https://issues.apache.org/jira/browse/AMBARI-22742 > > > Repository: ambari > > > Description > --- > > Server should raise events to publish created/updated upgrades to STOMP topic. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java > 1fbf1304135eabd11d7595060d660c38031afbb1 > > ambari-server/src/main/java/org/apache/ambari/server/events/AmbariUpdateEvent.java > cfd910b94bb10fd0ab961dd809313ef525c2b57e > > ambari-server/src/main/java/org/apache/ambari/server/events/DefaultMessageEmitter.java > 3ae5955d497955dba141d0af77701bdba5573fea > > ambari-server/src/main/java/org/apache/ambari/server/events/RequestUpdateEvent.java > 4133c6292b85bd37ca033b24e56c987da83862a0 > > ambari-server/src/main/java/org/apache/ambari/server/events/UpgradeUpdateEvent.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/events/listeners/upgrade/UpgradeUpdateListener.java > PRE-CREATION > > > Diff: https://reviews.apache.org/r/64990/diff/1/ > > > Testing > --- > > manual testing > > > Thanks, > > Myroslav Papirkovskyy > >
Re: Review Request 64989: AMBARI-22740 : Fix integration test for HBase in branch-3.0-ams due to UUID changes
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64989/#review194902 --- Ship it! Ship It! - Sid Wagle On Jan. 5, 2018, 8:16 p.m., Aravindan Vijayan wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64989/ > --- > > (Updated Jan. 5, 2018, 8:16 p.m.) > > > Review request for Ambari, Dmytro Sen and Sid Wagle. > > > Bugs: AMBARI-22740 > https://issues.apache.org/jira/browse/AMBARI-22740 > > > Repository: ambari > > > Description > --- > > Fix integration test for HBase in branch-3.0-ams due to UUID changes > > > Diffs > - > > ambari-metrics/ambari-metrics-timelineservice/pom.xml d3a976b > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java > 5b04c37 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricClusterAggregator.java > 7368bfb > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/discovery/TimelineMetricMetadataManager.java > 6b926ac > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/PhoenixTransactSQL.java > 1482f7c > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/uuid/HashBasedUuidGenStrategy.java > 10e9c61 > > ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/AbstractMiniHBaseClusterTest.java > c4cebd6 > > ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/ITPhoenixHBaseAccessor.java > 2a5dd0b > > ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/ITClusterAggregator.java > 1c5f41f > > ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/ITMetricAggregator.java > 1890819 > ambari-metrics/pom.xml 6541cec > > > Diff: https://reviews.apache.org/r/64989/diff/1/ > > > Testing > --- > > Unit testing. > > [INFO] > > [INFO] Reactor Summary: > [INFO] > [INFO] ambari-utility . SUCCESS [ 7.231 > s] > [INFO] Ambari Metrics Common .. SUCCESS [ 6.238 > s] > [INFO] Ambari Metrics Collector ... SUCCESS [05:19 > min] > [INFO] > > [INFO] BUILD SUCCESS > [INFO] > > [INFO] Total time: 05:33 min > [INFO] Finished at: 2018-01-05T12:13:47-08:00 > [INFO] Final Memory: 91M/1293M > [INFO] > > > > Thanks, > > Aravindan Vijayan > >
Re: Review Request 64991: Fix issues with webapp deployment with new Hadoop common changes
> On Jan. 5, 2018, 9:04 p.m., Aravindan Vijayan wrote: > > ambari-metrics/pom.xml > > Line 44 (original), 44 (patched) > > <https://reviews.apache.org/r/64991/diff/1/?file=1932962#file1932962line44> > > > > Extra ' character at the end. Great catch, thanks. - Sid --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64991/#review194898 ------- On Jan. 5, 2018, 8:16 p.m., Sid Wagle wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64991/ > --- > > (Updated Jan. 5, 2018, 8:16 p.m.) > > > Review request for Ambari and Aravindan Vijayan. > > > Bugs: AMBARI-22744 > https://issues.apache.org/jira/browse/AMBARI-22744 > > > Repository: ambari > > > Description > --- > > - Fix deploy time issues after migration. > - Fix imports. > > > Diffs > - > > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/AMSApplicationServer.java > f576362 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/loadsimulator/LoadRunner.java > cd1dd1b > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/loadsimulator/MetricsLoadSimulator.java > b64f542 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/loadsimulator/MetricsSenderWorker.java > d111eb6 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/loadsimulator/data/HostMetricsGenerator.java > 61a6624 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/loadsimulator/data/MetricsGeneratorConfigurer.java > b3401b2 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/loadsimulator/net/RestMetricsSender.java > 32af851 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/loadsimulator/util/Json.java > 61a3903 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricsService.java > d557974 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java > 5b04c37 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricConfiguration.java > d03e244 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricDistributedCache.java > 3480545 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricStore.java > b2cd1c2 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricStoreWatcher.java > 517d3a4 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricsIgniteCache.java > 6441c9c > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/AbstractTimelineAggregator.java > ab36605 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/DownSamplerUtils.java > b320e47 > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricAggregatorFactory.java > 9e493ea > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricAppAggregator.java > 09fbe81 > > ambari-metrics/ambari-metrics-timelineserv
Review Request 64991: Fix issues with webapp deployment with new Hadoop common changes
/r/64991/diff/1/ Testing --- Manually verified. Thanks, Sid Wagle
Re: Review Request 64984: /events/hostcomponents topic should react on maintenance and stale configs events.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64984/#review194894 --- Ship it! Makes sense, can you please open a refactor Jira so we do not loose this thread. - Sid Wagle On Jan. 5, 2018, 6:59 p.m., Myroslav Papirkovskyy wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64984/ > --- > > (Updated Jan. 5, 2018, 6:59 p.m.) > > > Review request for Ambari, Andrew Onischuk, Dmytro Grinenko, Eugene > Chekanskiy, and Sid Wagle. > > > Bugs: AMBARI-22737 > https://issues.apache.org/jira/browse/AMBARI-22737 > > > Repository: ambari > > > Description > --- > > /events/hostcomponents topic should react on maintenance state on/off and > stale configs events. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/agent/HeartbeatProcessor.java > fa697278c34d40afc12b93090a4663543bd94923 > > ambari-server/src/main/java/org/apache/ambari/server/events/AmbariEvent.java > 5f410f563c8bc1c167f72dd640b7c84a639a7040 > > ambari-server/src/main/java/org/apache/ambari/server/events/HostComponentUpdate.java > e48014654c127341f25f5f79ec67b93d973c3d5b > > ambari-server/src/main/java/org/apache/ambari/server/events/StaleConfigsUpdateEvent.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/events/listeners/hostcomponents/HostComponentsUpdateListener.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/events/listeners/hosts/HostUpdateListener.java > 6cb831f153e5748d19be26d4f899522fe9420ed6 > ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java > 7cc5799251913dd29ffb83ce13bf7ec328cab3a5 > > ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java > b5ade0c841009235c8a2cee593ca8f992902e4ec > > ambari-server/src/main/java/org/apache/ambari/server/state/ServiceComponentHost.java > 7f1c746eda955edcc3503595789d2bdc0b50891d > > ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java > d3ba1d50f8d7356d1445637fdc27d00b715de20e > > ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java > ef7d471affd191400e5ccc259420ae4c544789f8 > > ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerTest.java > a1f28f1b3a5dfc5e30b5a2348b681cca892746ba > > ambari-server/src/test/java/org/apache/ambari/server/controller/RefreshYarnCapacitySchedulerReleaseConfigTest.java > ef186f1ac02fa88f035821e073fdcdd6c0a3cada > > ambari-server/src/test/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostTest.java > 48864cc3baab7f7b0887f7dd68a5bbdcd9925698 > > > Diff: https://reviews.apache.org/r/64984/diff/1/ > > > Testing > --- > > manual testing > > > Thanks, > > Myroslav Papirkovskyy > >
Re: Review Request 64984: /events/hostcomponents topic should react on maintenance and stale configs events.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64984/#review194888 --- ambari-server/src/main/java/org/apache/ambari/server/agent/HeartbeatProcessor.java Lines 480 (patched) <https://reviews.apache.org/r/64984/#comment273966> Why is this needed to be done on HB processing? Can you please remind me if we still piggy back tags on HB, in any case shouldn't tags matching determine staleness? - Sid Wagle On Jan. 5, 2018, 6:59 p.m., Myroslav Papirkovskyy wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64984/ > --- > > (Updated Jan. 5, 2018, 6:59 p.m.) > > > Review request for Ambari, Andrew Onischuk, Dmytro Grinenko, Eugene > Chekanskiy, and Sid Wagle. > > > Bugs: AMBARI-22737 > https://issues.apache.org/jira/browse/AMBARI-22737 > > > Repository: ambari > > > Description > --- > > /events/hostcomponents topic should react on maintenance state on/off and > stale configs events. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/agent/HeartbeatProcessor.java > fa697278c34d40afc12b93090a4663543bd94923 > > ambari-server/src/main/java/org/apache/ambari/server/events/AmbariEvent.java > 5f410f563c8bc1c167f72dd640b7c84a639a7040 > > ambari-server/src/main/java/org/apache/ambari/server/events/HostComponentUpdate.java > e48014654c127341f25f5f79ec67b93d973c3d5b > > ambari-server/src/main/java/org/apache/ambari/server/events/StaleConfigsUpdateEvent.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/events/listeners/hostcomponents/HostComponentsUpdateListener.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/events/listeners/hosts/HostUpdateListener.java > 6cb831f153e5748d19be26d4f899522fe9420ed6 > ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java > 7cc5799251913dd29ffb83ce13bf7ec328cab3a5 > > ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java > b5ade0c841009235c8a2cee593ca8f992902e4ec > > ambari-server/src/main/java/org/apache/ambari/server/state/ServiceComponentHost.java > 7f1c746eda955edcc3503595789d2bdc0b50891d > > ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java > d3ba1d50f8d7356d1445637fdc27d00b715de20e > > ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java > ef7d471affd191400e5ccc259420ae4c544789f8 > > ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerTest.java > a1f28f1b3a5dfc5e30b5a2348b681cca892746ba > > ambari-server/src/test/java/org/apache/ambari/server/controller/RefreshYarnCapacitySchedulerReleaseConfigTest.java > ef186f1ac02fa88f035821e073fdcdd6c0a3cada > > ambari-server/src/test/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostTest.java > 48864cc3baab7f7b0887f7dd68a5bbdcd9925698 > > > Diff: https://reviews.apache.org/r/64984/diff/1/ > > > Testing > --- > > manual testing > > > Thanks, > > Myroslav Papirkovskyy > >
Re: Review Request 64983: Setup heartbeat for api endpoint.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64983/#review194879 --- Ship it! Ship It! - Sid Wagle On Jan. 5, 2018, 5:44 p.m., Myroslav Papirkovskyy wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64983/ > --- > > (Updated Jan. 5, 2018, 5:44 p.m.) > > > Review request for Ambari, Andrew Onischuk, Dmytro Grinenko, Eugene > Chekanskiy, and Sid Wagle. > > > Bugs: AMBARI-22738 > https://issues.apache.org/jira/browse/AMBARI-22738 > > > Repository: ambari > > > Description > --- > > API endpoint uses sockJS fallback options. The SockJS protocol requires > server to set proper heartbeat configuration. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java > 4dffcaa23ab6a7ca5e2f558ae2bbc3ba89d4cbc4 > > ambari-server/src/main/java/org/apache/ambari/server/configuration/spring/ApiStompConfig.java > e968b11f1ae22631cc656003b98ce1da83b25cee > > > Diff: https://reviews.apache.org/r/64983/diff/1/ > > > Testing > --- > > manual testing > > > Thanks, > > Myroslav Papirkovskyy > >
Re: Review Request 64789: Fix AMS compilation issues and unit test with hbase, hadoop and phoenix upgraded
/ApplicationHistoryStoreTestUtils.java ec9b49d ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/TestApplicationHistoryClientService.java f93ac5e ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/TestApplicationHistoryManagerImpl.java aad23d9 ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/TestApplicationHistoryServer.java 03205e7 ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/TestFileSystemApplicationHistoryStore.java 543c25b ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/TestMemoryApplicationHistoryStore.java b4da01a ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/AbstractMiniHBaseClusterTest.java 741bb3c ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/ITPhoenixHBaseAccessor.java 57f9796 ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/timeline/TestLeveldbTimelineStore.java 9b27309 ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/timeline/TestMemoryTimelineStore.java 415de53 ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/timeline/TimelineStoreTestUtils.java d760536 ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/webapp/TestAHSWebApp.java 605358f ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/webapp/TestAHSWebServices.java 44b3f65 ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/webapp/TestTimelineWebServices.java b093a2a ambari-metrics/pom.xml ff81060 Diff: https://reviews.apache.org/r/64789/diff/2/ Changes: https://reviews.apache.org/r/64789/diff/1-2/ Testing --- Unit tests passed. [INFO] [INFO] Reactor Summary: [INFO] [INFO] ambari-utility . SUCCESS [ 2.751 s] [INFO] Ambari Metrics Common .. SUCCESS [ 1.390 s] [INFO] Ambari Metrics Collector ... SUCCESS [01:43 min] [INFO] [INFO] BUILD SUCCESS [INFO] [INFO] Total time: 01:48 min [INFO] Finished at: 2017-12-21T14:23:33-08:00 [INFO] Final Memory: 94M/600M [INFO] Thanks, Sid Wagle
Review Request 64789: Fix AMS compilation issues and unit test with hbase, hadoop and phoenix upgraded
/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/webapp/NavBlock.java e84ddec ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/webapp/TimelineWebServices.java dc401e6 ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryStoreTestUtils.java ec9b49d ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/TestApplicationHistoryClientService.java f93ac5e ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/TestApplicationHistoryManagerImpl.java aad23d9 ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/TestApplicationHistoryServer.java 03205e7 ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/TestFileSystemApplicationHistoryStore.java 543c25b ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/TestMemoryApplicationHistoryStore.java b4da01a ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/AbstractMiniHBaseClusterTest.java 741bb3c ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/ITPhoenixHBaseAccessor.java 57f9796 ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/timeline/TestLeveldbTimelineStore.java 9b27309 ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/timeline/TestMemoryTimelineStore.java 415de53 ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/timeline/TimelineStoreTestUtils.java d760536 ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/webapp/TestAHSWebApp.java 605358f ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/webapp/TestAHSWebServices.java 44b3f65 ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/webapp/TestTimelineWebServices.java b093a2a ambari-metrics/pom.xml ff81060 Diff: https://reviews.apache.org/r/64789/diff/1/ Testing --- Unit tests passed. [INFO] [INFO] Reactor Summary: [INFO] [INFO] ambari-utility . SUCCESS [ 2.751 s] [INFO] Ambari Metrics Common .. SUCCESS [ 1.390 s] [INFO] Ambari Metrics Collector ... SUCCESS [01:43 min] [INFO] [INFO] BUILD SUCCESS [INFO] [INFO] Total time: 01:48 min [INFO] Finished at: 2017-12-21T14:23:33-08:00 [INFO] Final Memory: 94M/600M [INFO] Thanks, Sid Wagle
Re: Review Request 64607: Ambari Infra Manager: solr data exporting jobs and integration test environment
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64607/#review194364 --- Ship it! Ship It! - Sid Wagle On Dec. 15, 2017, 8:57 a.m., Krisztian Kasa wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64607/ > --- > > (Updated Dec. 15, 2017, 8:57 a.m.) > > > Review request for Ambari, Miklos Gergely, Oliver Szabo, and Sid Wagle. > > > Bugs: AMBARI-22514 and AMBARI-22653 > https://issues.apache.org/jira/browse/AMBARI-22514 > https://issues.apache.org/jira/browse/AMBARI-22653 > > > Repository: ambari > > > Description > --- > > * Integration test environment for ambari-infra-manager using docker-compose > and jbehave > * test cases for testing data export jobs > > * deleting temporary files as soon as the file becomes unnecessary > * configuring solr data export like jobs in property files only, no hardcoded > job and step configurations: query can be defined in property file for > pulling data from solr > * validate job property values at application startup and before job execution > * s3 upload support > * not just interval end but interval start can be specified when exporting > * interval start and end values can be other type than dateTime > > > Diffs > - > > ambari-infra/ambari-infra-manager-it/pom.xml PRE-CREATION > > ambari-infra/ambari-infra-manager-it/src/test/java/org/apache/ambari/infra/InfraClient.java > PRE-CREATION > > ambari-infra/ambari-infra-manager-it/src/test/java/org/apache/ambari/infra/InfraManagerStories.java > PRE-CREATION > > ambari-infra/ambari-infra-manager-it/src/test/java/org/apache/ambari/infra/OffsetDateTimeConverter.java > PRE-CREATION > > ambari-infra/ambari-infra-manager-it/src/test/java/org/apache/ambari/infra/steps/AbstractInfraSteps.java > PRE-CREATION > > ambari-infra/ambari-infra-manager-it/src/test/java/org/apache/ambari/infra/steps/ExportJobsSteps.java > PRE-CREATION > ambari-infra/ambari-infra-manager-it/src/test/resources/log4j.properties > PRE-CREATION > > ambari-infra/ambari-infra-manager-it/src/test/resources/stories/infra_api_tests.story > PRE-CREATION > ambari-infra/ambari-infra-manager/docker/Dockerfile adb584acec > ambari-infra/ambari-infra-manager/docker/docker-compose.yml PRE-CREATION > ambari-infra/ambari-infra-manager/docker/infra-manager-docker-compose.sh > PRE-CREATION > > ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/ObjectSource.java > PRE-CREATION > > ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/archive/AbstractFileAction.java > PRE-CREATION > > ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/archive/CompositeFileAction.java > 84ce16074b > > ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/archive/Document.java > 84f5ece26f > > ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/archive/DocumentExportConfiguration.java > 69f41d333e > > ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/archive/DocumentExportJobListener.java > f1df46cfcc > > ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/archive/DocumentExportProperties.java > d6301c0c0e > > ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/archive/DocumentExportPropertyMap.java > PRE-CREATION > > ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/archive/DocumentExportStepListener.java > 3bab6d505e > > ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/archive/DocumentItemReader.java > a4378a4340 > > ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/archive/DocumentIterator.java > 6232cfc4c6 > > ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/archive/DocumentSource.java > c9871a3660 > > ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/archive/FileAction.java > 26a8c6310f > > ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/archive/LocalDocumentItemWriter.java > 02d898d979 > > ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/archive/S3Properties.java > 495401d633 > > a
Review Request 64785: Disabled stack still appears in the UI if VDF is available
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64785/ --- Review request for Ambari, Aravindan Vijayan, Jonathan Hurley, and Nate Cole. Bugs: AMBARI-22686 https://issues.apache.org/jira/browse/AMBARI-22686 Repository: ambari Description --- Ambari-2.6.1, Ambari UI still shows HDP-2.3 as a stack that is active although the stack definition marks this as disabled. Seems like the UI does not make any calls to check stack active/in-active. The stacks returned from VDF endpoint are then used to get the repo URLs because the VDF returns the stack def as sub-resource which included the in-active stack. Diffs - ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java 4ac8574 ambari-server/src/test/java/org/apache/ambari/server/api/services/AmbariMetaInfoTest.java e9567f1 ambari-server/src/test/java/org/apache/ambari/server/controller/internal/RepositoryVersionResourceProviderTest.java 6bc8b95 Diff: https://reviews.apache.org/r/64785/diff/1/ Testing --- Added a unit test. Verified manually. Thanks, Sid Wagle
Re: Review Request 64553: Log Feeder refactor: integrate with spring boot
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64553/#review193736 --- Ship it! ambari-logsearch/ambari-logsearch-config-api/src/main/java/org/apache/ambari/logsearch/config/api/LogSearchConfigFactory.java Lines 40 (patched) <https://reviews.apache.org/r/64553/#comment272360> When would this be false? ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/metrics/StatsLogger.java Lines 30 (patched) <https://reviews.apache.org/r/64553/#comment272362> More contemporary design would be to create a Runnable that can be used by the caller as an argument scheduledExectuor. The scehduling logic can be kept out of the class. - Sid Wagle On Dec. 13, 2017, 3:24 p.m., Oliver Szabo wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64553/ > --- > > (Updated Dec. 13, 2017, 3:24 p.m.) > > > Review request for Ambari, Krisztian Kasa and Sid Wagle. > > > Bugs: AMBARI-22639 > https://issues.apache.org/jira/browse/AMBARI-22639 > > > Repository: ambari > > > Description > --- > > First patch for refactor logfeeder - Spring Boot support > > Add spring boot support, the goal is to get rid of LogFeederPropertiesUtil > and let Spring to handle configs & beans. > - remove LogFeederPropertiesUtil > - move logfeeder properties fields into mutliple POJOs > - update some tests & remove one test (needs to rewrite it) > - convert singleton classes to spring beans > - let Spring to handle application pid file > - convert some internal threads to classes (like StatsLogger) > > (it does not contain any real changes regarding to input-filter-output blocks) > > Next patch will be focus on the usage of start scritps (in ambari-logsearch > and ambari-server stack code) > > > Diffs > - > > > ambari-logsearch/ambari-logsearch-config-api/src/main/java/org/apache/ambari/logsearch/config/api/LogSearchConfigFactory.java > a84a97b > > ambari-logsearch/ambari-logsearch-config-api/src/main/java/org/apache/ambari/logsearch/config/api/LogSearchConfigLogFeeder.java > 6ed36fd > ambari-logsearch/ambari-logsearch-logfeeder/.gitignore PRE-CREATION > ambari-logsearch/ambari-logsearch-logfeeder/pom.xml 01710bf > > ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/LogFeeder.java > 5114743 > > ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/LogFeederCommandLine.java > d996f98 > > ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/common/ConfigHandler.java > 243b344 > > ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/common/ConfigItem.java > 5c20a8e > > ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/common/LogEntryParseTester.java > ec29f69 > > ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/common/LogFeederConstants.java > a76 > > ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/conf/ApplicationConfig.java > PRE-CREATION > > ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/conf/InputSimulateConfig.java > PRE-CREATION > > ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/conf/LogEntryCacheConfig.java > PRE-CREATION > > ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/conf/LogFeederProps.java > PRE-CREATION > > ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/conf/LogFeederSecurityConfig.java > PRE-CREATION > > ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/conf/MetricsCollectorConfig.java > PRE-CREATION > > ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/filter/Filter.java > 8e8834b > > ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/filter/FilterGrok.java > fc7a565 > > ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/filter/FilterKeyValue.java > 8e5aee8 > > ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/AbstractInputFile.java > b021c37 > >
Re: Review Request 64537: AMBARI-22635 : Ambari should create a dummy core-site.xml for Ranger plugins when namenode is not installed.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64537/#review193734 --- Ship it! Ship It! - Sid Wagle On Dec. 13, 2017, 3:42 p.m., Vishal Suvagia wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64537/ > --- > > (Updated Dec. 13, 2017, 3:42 p.m.) > > > Review request for Ambari, Gautam Borad, Jonathan Hurley, Jayush Luniya, > Madhuvanthi Radhakrishnan, Mugdha Varadkar, and Velmurugan Periasamy. > > > Bugs: AMBARI-22635 > https://issues.apache.org/jira/browse/AMBARI-22635 > > > Repository: ambari > > > Description > --- > > For Ranger plugins to work properly in kerberised environments where HDFS is > not installed. We need to create a core-site.xml for Storm and Kafka plugins > so that the plugins can work to fetch latest policies from with kerberised > calls from Ranger. > Attached patch is for trunk branch, seperate patch for branch-2.6 is attached > to Apache JIRA. > > > Diffs > - > > > ambari-common/src/main/python/resource_management/libraries/functions/setup_ranger_plugin_xml.py > 485c1a6 > > ambari-server/src/main/resources/common-services/KAFKA/0.10.0.3.0/package/scripts/setup_ranger_kafka.py > e9719aa > > ambari-server/src/main/resources/common-services/KAFKA/0.8.1/package/scripts/setup_ranger_kafka.py > 9aa09df > > ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/setup_ranger_knox.py > c486ef7 > > ambari-server/src/main/resources/common-services/KNOX/0.5.0.3.0/package/scripts/setup_ranger_knox.py > c486ef7 > > ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/setup_ranger_storm.py > c04496e > > ambari-server/src/main/resources/common-services/STORM/1.0.1.3.0/package/scripts/setup_ranger_storm.py > c04496e > > > Diff: https://reviews.apache.org/r/64537/diff/2/ > > > Testing > --- > > Verified with installation on Cent-OS 6. > > > Thanks, > > Vishal Suvagia > >
Re: Review Request 64537: AMBARI-22635 : Ambari should create a dummy core-site.xml for Ranger plugins when namenode is not installed.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64537/#review193559 --- Instead of putting empty files on the classpath isn't it easier to check for file existence on plugin side and defaul to "simple" auth? - Sid Wagle On Dec. 12, 2017, 12:35 p.m., Vishal Suvagia wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64537/ > --- > > (Updated Dec. 12, 2017, 12:35 p.m.) > > > Review request for Ambari, Gautam Borad, Jonathan Hurley, Jayush Luniya, > Madhuvanthi Radhakrishnan, Mugdha Varadkar, and Velmurugan Periasamy. > > > Bugs: AMBARI-22635 > https://issues.apache.org/jira/browse/AMBARI-22635 > > > Repository: ambari > > > Description > --- > > For Ranger plugins to work properly in kerberised environments where HDFS is > not installed. We need to create a core-site.xml for Storm and Kafka plugins > so that the plugins can work to fetch latest policies from with kerberised > calls from Ranger. > Attached patch is for trunk branch, seperate patch for branch-2.6 is attached > to Apache JIRA. > > > Diffs > - > > > ambari-server/src/main/resources/common-services/KAFKA/0.10.0.3.0/package/scripts/setup_ranger_kafka.py > e9719aa > > ambari-server/src/main/resources/common-services/KAFKA/0.8.1/package/scripts/setup_ranger_kafka.py > 9aa09df > > ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/setup_ranger_storm.py > c04496e > > ambari-server/src/main/resources/common-services/STORM/1.0.1.3.0/package/scripts/setup_ranger_storm.py > c04496e > > > Diff: https://reviews.apache.org/r/64537/diff/1/ > > > Testing > --- > > Verified with installation on Cent-OS 6. > > > Thanks, > > Vishal Suvagia > >
Re: Review Request 64513: Update the default storm log search configuration (use wildcards)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64513/#review193556 --- Last revision has noly whitespace changes? - Sid Wagle On Dec. 12, 2017, 4:19 p.m., Oliver Szabo wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64513/ > --- > > (Updated Dec. 12, 2017, 4:19 p.m.) > > > Review request for Ambari, Krisztian Kasa, Miklos Gergely, and Sid Wagle. > > > Bugs: AMBARI-22630 > https://issues.apache.org/jira/browse/AMBARI-22630 > > > Repository: ambari > > > Description > --- > > set the storm config defaults for logseach as other teams needs it. > > this patch is for branch-2.6, but if cannot be delivered in 2.6.1, that can > be used in a hotfix > > > Diffs > - > > > ambari-server/src/main/resources/common-services/STORM/0.9.1/configuration/storm-logsearch-conf.xml > d485017 > > > Diff: https://reviews.apache.org/r/64513/diff/2/ > > > Testing > --- > > not yet. > > > Thanks, > > Oliver Szabo > >
Re: Review Request 64513: Update the default storm log search configuration (use wildcards)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64513/#review193555 --- Ship it! Ship It! - Sid Wagle On Dec. 11, 2017, 11:47 p.m., Oliver Szabo wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64513/ > --- > > (Updated Dec. 11, 2017, 11:47 p.m.) > > > Review request for Ambari, Krisztian Kasa, Miklos Gergely, and Sid Wagle. > > > Bugs: AMBARI-22630 > https://issues.apache.org/jira/browse/AMBARI-22630 > > > Repository: ambari > > > Description > --- > > set the storm config defaults for logseach as other teams needs it. > > this patch is for branch-2.6, but if cannot be delivered in 2.6.1, that can > be used in a hotfix > > > Diffs > - > > > ambari-server/src/main/resources/common-services/STORM/0.9.1/configuration/storm-logsearch-conf.xml > d485017 > > > Diff: https://reviews.apache.org/r/64513/diff/1/ > > > Testing > --- > > not yet. > > > Thanks, > > Oliver Szabo > >
Re: Review Request 64528: Zeppelin Interpreter settings are getting updated after zeppelin restart
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64528/#review193554 --- Ship it! Ship It! - Sid Wagle On Dec. 12, 2017, 7:39 a.m., Prabhjyot Singh wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64528/ > --- > > (Updated Dec. 12, 2017, 7:39 a.m.) > > > Review request for Ambari, DIPAYAN BHOWMICK, Jaimin Jetly, Jayush Luniya, > Rohit Choudhary, Sumit Mohanty, and venkat sairam. > > > Bugs: AMBARI-22626 > https://issues.apache.org/jira/browse/AMBARI-22626 > > > Repository: ambari > > > Description > --- > > Zeppelin Interpreter settings are getting updated after zeppelin restart. > > Steps to reproduce : > 1) Update the zeppelin.pyspark.python to /base/tools/python-2.7.14/bin/python > for spark2 interpreter > 2) Restart zeppelin > 3) After restart zeppelin.pyspark.python is getting overridden with value > 'python'. > Same is observed with livy2 interpreter. > > But same steps for spark interpreter is working fine even after zeppelin > restart. > > So looks like issue is happening only for spark2 and livy2 interpreter. > > > Diffs > - > > > ambari-server/src/main/resources/common-services/ZEPPELIN/0.7.0/package/scripts/interpreter_json_template.py > 713db232b1 > > ambari-server/src/main/resources/common-services/ZEPPELIN/0.7.0/package/scripts/livy2_config_template.py > c9030bd33e > > ambari-server/src/main/resources/common-services/ZEPPELIN/0.7.0/package/scripts/master.py > bb4c5b227b > > ambari-server/src/main/resources/common-services/ZEPPELIN/0.7.0/package/scripts/spark2_config_template.py > 28a63c6f1b > > ambari-server/src/test/python/stacks/2.6/ZEPPELIN/interpreter_json_generated.py > 4b4dc1fe48 > > > Diff: https://reviews.apache.org/r/64528/diff/1/ > > > Testing > --- > > Manually > > > Thanks, > > Prabhjyot Singh > >
Re: Review Request 64393: AMBARI-22607 : Design and implement an AD job scheduler.
> On Dec. 7, 2017, 3:03 a.m., Sid Wagle wrote: > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/configuration/DetectionServiceConfiguration.scala > > Lines 39 (patched) > > <https://reviews.apache.org/r/64393/diff/1/?file=1909974#file1909974line39> > > > > Description / comments? > > Aravindan Vijayan wrote: > Will refine this in when we work on EMA application. For this patch, I > used EMA as a simple example job. Sure go ahead and close issues that will be addresses later on, just want to make sure the point is raised so we don't forget. - Sid --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64393/#review193087 --- On Dec. 6, 2017, 10:52 p.m., Aravindan Vijayan wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64393/ > --- > > (Updated Dec. 6, 2017, 10:52 p.m.) > > > Review request for Ambari, Jayush Luniya and Sid Wagle. > > > Bugs: AMBARI-22607 > https://issues.apache.org/jira/browse/AMBARI-22607 > > > Repository: ambari > > > Description > --- > > Design and implement a scheduler for starting and managing AD jobs on Spark. > The Spark runtime configuration will be analyzed and refined in follow up > JIRAs. > > > Diffs > - > > > ambari-metrics/ambari-metrics-anomaly-detection-service/conf/unix/config.yaml > 85e4004 > ambari-metrics/ambari-metrics-anomaly-detection-service/pom.xml 50d7ef6 > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/java/org/apache/ambari/metrics/adservice/prototype/core/MetricSparkConsumer.java > addeda7 > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/app/AnomalyDetectionApp.scala > 2d0dbdf > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/app/AnomalyDetectionAppConfig.scala > 58efa97 > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/app/AnomalyDetectionAppModule.scala > 68e9df9 > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/configuration/DetectionServiceConfiguration.scala > PRE-CREATION > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/configuration/SparkConfiguration.scala > 30efdc7 > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/db/PhoenixAnomalyStoreAccessor.scala > 53e6dee > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/detection/AdJobManager.scala > PRE-CREATION > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/detection/SparkApplicationRunner.scala > PRE-CREATION > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/detection/Subsystem.scala > PRE-CREATION > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/detection/pointintime/EmaSparkDriver.scala > PRE-CREATION > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/detection/pointintime/PointInTimeSubsystem.scala > PRE-CREATION > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/metadata/MetricDefinitionService.scala > 52ce39e > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/metadata/MetricDefinitionServiceImpl.scala > b9b4a7c > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/metadata/MetricSourceDefinition.scala > 47b1499 > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/resource/DetectionResource.scala > PRE-CREATION > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/resource/MetricDefinitionResource.scala > 442bf46 > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/
Re: Review Request 64393: AMBARI-22607 : Design and implement an AD job scheduler.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64393/#review193087 --- Fix it, then Ship it! ambari-metrics/ambari-metrics-anomaly-detection-service/pom.xml Lines 244 (patched) <https://reviews.apache.org/r/64393/#comment271574> 2.11 is binary verison - parameterize ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/java/org/apache/ambari/metrics/adservice/prototype/core/MetricSparkConsumer.java Line 155 (original), 154 (patched) <https://reviews.apache.org/r/64393/#comment271575> Can move this to a unit test? ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/configuration/DetectionServiceConfiguration.scala Lines 39 (patched) <https://reviews.apache.org/r/64393/#comment271576> Description / comments? ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/detection/AdJobManager.scala Lines 80 (patched) <https://reviews.apache.org/r/64393/#comment271577> Nice !! ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/detection/pointintime/EmaSparkDriver.scala Lines 79 (patched) <https://reviews.apache.org/r/64393/#comment271579> Debugging? ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/detection/pointintime/PointInTimeSubsystem.scala Lines 61 (patched) <https://reviews.apache.org/r/64393/#comment271580> Better model will be to add a listener and handle events. - Sid Wagle On Dec. 6, 2017, 10:52 p.m., Aravindan Vijayan wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64393/ > --- > > (Updated Dec. 6, 2017, 10:52 p.m.) > > > Review request for Ambari, Jayush Luniya and Sid Wagle. > > > Bugs: AMBARI-22607 > https://issues.apache.org/jira/browse/AMBARI-22607 > > > Repository: ambari > > > Description > --- > > Design and implement a scheduler for starting and managing AD jobs on Spark. > The Spark runtime configuration will be analyzed and refined in follow up > JIRAs. > > > Diffs > - > > > ambari-metrics/ambari-metrics-anomaly-detection-service/conf/unix/config.yaml > 85e4004 > ambari-metrics/ambari-metrics-anomaly-detection-service/pom.xml 50d7ef6 > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/java/org/apache/ambari/metrics/adservice/prototype/core/MetricSparkConsumer.java > addeda7 > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/app/AnomalyDetectionApp.scala > 2d0dbdf > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/app/AnomalyDetectionAppConfig.scala > 58efa97 > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/app/AnomalyDetectionAppModule.scala > 68e9df9 > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/configuration/DetectionServiceConfiguration.scala > PRE-CREATION > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/configuration/SparkConfiguration.scala > 30efdc7 > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/db/PhoenixAnomalyStoreAccessor.scala > 53e6dee > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/detection/AdJobManager.scala > PRE-CREATION > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/detection/SparkApplicationRunner.scala > PRE-CREATION > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/detection/Subsystem.scala > PRE-CREATION > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/detection/pointintime/EmaSparkDriver.scala > PRE-CREATION > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/detection/pointintime/PointInTimeSubsystem.scala > PRE-CREATION > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adserv
Re: Review Request 64373: A bad WebHDFS request is issued when starting Hive Metastore
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64373/#review193041 --- Ship it! Ship It! - Sid Wagle On Dec. 6, 2017, 1:04 p.m., Attila Magyar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64373/ > --- > > (Updated Dec. 6, 2017, 1:04 p.m.) > > > Review request for Ambari, Eugene Chekanskiy, Robert Levas, Sandor Molnar, > and Sid Wagle. > > > Bugs: AMBARI-22492 > https://issues.apache.org/jira/browse/AMBARI-22492 > > > Repository: ambari > > > Description > --- > > The default values of hive_repl_cmrootdir, hive_repl_rootdir are empty > string. The empty string can pass through the existing "is None" check which > will cause the webhdfsresource to build an incorrect url. > > > Diffs > - > > > ambari-common/src/main/python/resource_management/libraries/providers/hdfs_resource.py > 0c45719 > > ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive.py > 603bc92 > > > Diff: https://reviews.apache.org/r/64373/diff/1/ > > > Testing > --- > > Tested by starting hiveserver2. > > > existing tests: > > Ran 276 tests in 7.399s > OK > -- > Total run:1200 > Tests run: 4932, Failures: 0, Errors: 0, Skipped: 34 > -- > Ran 455 tests in 97.752s > > > Thanks, > > Attila Magyar > >
Re: Review Request 64349: LogFeeder: filter objects for wildcard input paths need to be cloned
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64349/#review192963 --- Ship it! Ok good for now. This area definitely seems like it needs some rewrite to use Executors and better concurrency contructs but for branch-2.6 adding a +1 - Sid Wagle On Dec. 5, 2017, 8:50 p.m., Oliver Szabo wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64349/ > --- > > (Updated Dec. 5, 2017, 8:50 p.m.) > > > Review request for Ambari, Krisztian Kasa, Miklos Gergely, and Robert > Nettleton. > > > Bugs: AMBARI-22600 > https://issues.apache.org/jira/browse/AMBARI-22600 > > > Repository: ambari > > > Description > --- > > Log Feeder needs to clone filters for different input threads (for wildcard > pattern). > > As different threads tried to access the same grok filter object (mutable) > that could cause to some docs used the wrong path (or other fields), > especially when a grok error happened. (on error, in the code, we checked > some objects regarding that what to do, but other threads changed those > objects) > > Also some other changes: > - use sdi_* dynamic fields > - add new input log files > - add new config to grok filter: skip_on_error (can be useful for filter > chains) > > > Diffs > - > > > ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/filter/Filter.java > afd903e > > ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/filter/FilterGrok.java > 49d7e76 > > ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/Input.java > 7df0b6e > > ambari-logsearch/docker/test-config/logfeeder/shipper-conf/input.config-storm.json > 34bdcf0 > > ambari-logsearch/docker/test-logs/storm/worker-logs/streamline-1-TestAgg-2-3/6701/worker.log > 6a10ad9 > > ambari-logsearch/docker/test-logs/storm/worker-logs/streamline-2-TestAgg2-4-5/6700/worker.log > e69de29 > > > Diff: https://reviews.apache.org/r/64349/diff/1/ > > > Testing > --- > > done, i need to write some unit tests for the filter cloning. > > > Thanks, > > Oliver Szabo > >
Re: Review Request 64349: LogFeeder: filter objects for wildcard input paths need to be cloned
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64349/#review192930 --- ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/Input.java Lines 215 (patched) <https://reviews.apache.org/r/64349/#comment271281> Have you considered using AtomicReference here vs deepCloning? - Sid Wagle On Dec. 5, 2017, 8:50 p.m., Oliver Szabo wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64349/ > --- > > (Updated Dec. 5, 2017, 8:50 p.m.) > > > Review request for Ambari, Krisztian Kasa, Miklos Gergely, and Robert > Nettleton. > > > Bugs: AMBARI-22600 > https://issues.apache.org/jira/browse/AMBARI-22600 > > > Repository: ambari > > > Description > --- > > Log Feeder needs to clone filters for different input threads (for wildcard > pattern). > > As different threads tried to access the same grok filter object (mutable) > that could cause to some docs used the wrong path (or other fields), > especially when a grok error happened. (on error, in the code, we checked > some objects regarding that what to do, but other threads changed those > objects) > > Also some other changes: > - use sdi_* dynamic fields > - add new input log files > - add new config to grok filter: skip_on_error (can be useful for filter > chains) > > > Diffs > - > > > ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/filter/Filter.java > afd903e > > ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/filter/FilterGrok.java > 49d7e76 > > ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/Input.java > 7df0b6e > > ambari-logsearch/docker/test-config/logfeeder/shipper-conf/input.config-storm.json > 34bdcf0 > > ambari-logsearch/docker/test-logs/storm/worker-logs/streamline-1-TestAgg-2-3/6701/worker.log > 6a10ad9 > > ambari-logsearch/docker/test-logs/storm/worker-logs/streamline-2-TestAgg2-4-5/6700/worker.log > e69de29 > > > Diff: https://reviews.apache.org/r/64349/diff/1/ > > > Testing > --- > > done, i need to write some unit tests for the filter cloning. > > > Thanks, > > Oliver Szabo > >
Re: Review Request 64315: Hive Client restart fails: hadooplzo package not present in iop repos
> On Dec. 5, 2017, 4:56 p.m., Jonathan Hurley wrote:
> > ambari-common/src/main/python/resource_management/libraries/functions/lzo_utils.py
> > Lines 72 (patched)
> > <https://reviews.apache.org/r/64315/diff/2/?file=1909085#file1909085line75>
> >
> > Maybe also logger.info "This stack does not indicate that it supports
> > LZO installation"
Good point. Thanks for the quick review will make those changes before commit.
- Sid
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64315/#review192882
-----------
On Dec. 5, 2017, 4:51 p.m., Sid Wagle wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64315/
> ---
>
> (Updated Dec. 5, 2017, 4:51 p.m.)
>
>
> Review request for Ambari, Aravindan Vijayan and Jonathan Hurley.
>
>
> Bugs: AMBARI-22588
> https://issues.apache.org/jira/browse/AMBARI-22588
>
>
> Repository: ambari
>
>
> Description
> ---
>
> IOP Migration: Hive Client restart fails: hadooplzo package not present in
> iop repos
>
> *STR*:
> - Install IOP-425 cluster
> - Now upgrade Ambari to 2.6.1. Hive client restart fails
>
> {code}
> Cannot match package for regexp name hadooplzo_${stack_version}
> {code}
>
>
> Diffs
> -
>
>
> ambari-common/src/main/python/resource_management/libraries/functions/lzo_utils.py
> 68ee607
>
> ambari-server/src/main/resources/stacks/BigInsights/4.0/properties/stack_features.json
> d29efb2
>
>
> Diff: https://reviews.apache.org/r/64315/diff/2/
>
>
> Testing
> ---
>
> mvn clean test
>
>
> Thanks,
>
> Sid Wagle
>
>
Re: Review Request 64315: Hive Client restart fails: hadooplzo package not present in iop repos
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64315/
---
(Updated Dec. 5, 2017, 4:51 p.m.)
Review request for Ambari, Aravindan Vijayan and Jonathan Hurley.
Changes
---
Preliminary patch for review: Using stack_features instead of stack code change.
Bugs: AMBARI-22588
https://issues.apache.org/jira/browse/AMBARI-22588
Repository: ambari
Description
---
IOP Migration: Hive Client restart fails: hadooplzo package not present in iop
repos
*STR*:
- Install IOP-425 cluster
- Now upgrade Ambari to 2.6.1. Hive client restart fails
{code}
Cannot match package for regexp name hadooplzo_${stack_version}
{code}
Diffs (updated)
-
ambari-common/src/main/python/resource_management/libraries/functions/lzo_utils.py
68ee607
ambari-server/src/main/resources/stacks/BigInsights/4.0/properties/stack_features.json
d29efb2
Diff: https://reviews.apache.org/r/64315/diff/2/
Changes: https://reviews.apache.org/r/64315/diff/1-2/
Testing
---
mvn clean test
Thanks,
Sid Wagle
Re: Review Request 64209: Need to address HDP-GPL repo update after user accepts license in post-install scenario
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64209/#review192874 --- Ship it! Ship It! - Sid Wagle On Dec. 5, 2017, 11:04 a.m., Andrew Onischuk wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64209/ > --- > > (Updated Dec. 5, 2017, 11:04 a.m.) > > > Review request for Ambari, Attila Doroszlai, Myroslav Papirkovskyy, Nate > Cole, and Sid Wagle. > > > Bugs: AMBARI-22561 > https://issues.apache.org/jira/browse/AMBARI-22561 > > > Repository: ambari > > > Description > --- > > * User denies the GPL license agreement, UI will still issue PUT call to > create the repo > * Ambari should not write HDP-GPL repo info to the ambari-hdp-1.repo > because it will break yum in local repo world. > * Then the user accepts the license by running ambari-server setup > * The repo file does not get updated on existing hosts > * We need to push the HDP-GPL repo to the repo files on existing hosts only > when the license is accepted > > > Diffs > - > > > ambari-common/src/main/python/resource_management/libraries/functions/lzo_utils.py > 68ee607b9b > > ambari-common/src/main/python/resource_management/libraries/functions/repository_util.py > f1c8ef1b04 > > ambari-common/src/main/python/resource_management/libraries/script/script.py > ccb8a547f4 > > ambari-server/src/main/java/org/apache/ambari/server/agent/CommandRepository.java > e207ac5289 > ambari-server/src/main/resources/custom_actions/scripts/install_packages.py > f2218e5e81 > > ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/repo_initialization.py > 82e57aa4f4 > ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_server.py > 10a83d6c23 > ambari-server/src/test/python/stacks/2.2/configs/oozie-upgrade.json > 29cbddca55 > ambari-server/src/test/python/stacks/utils/RMFTestCase.py 80712c5ebe > > > Diff: https://reviews.apache.org/r/64209/diff/2/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Andrew Onischuk > >
Re: Review Request 64302: PERF 1.0 stack deployment failure in ambari-2.6.1.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64302/#review192725 --- Ship it! Ship It! - Sid Wagle On Dec. 4, 2017, 4:15 p.m., Myroslav Papirkovskyy wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64302/ > --- > > (Updated Dec. 4, 2017, 4:15 p.m.) > > > Review request for Ambari, Andrew Onischuk and Sid Wagle. > > > Bugs: AMBARI-22584 > https://issues.apache.org/jira/browse/AMBARI-22584 > > > Repository: ambari > > > Description > --- > > Backport of AMBARI-22520 > > > Diffs > - > > > ambari-server/src/main/resources/stacks/PERF/1.0/configuration/cluster-env.xml > f19ac5237e278841dcf1b01db963004753ab77fe > > ambari-server/src/main/resources/stacks/PERF/1.0/properties/stack_packages.json > PRE-CREATION > > > Diff: https://reviews.apache.org/r/64302/diff/1/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Myroslav Papirkovskyy > >
Re: Review Request 64209: Need to address HDP-GPL repo update after user accepts license in post-install scenario
> On Nov. 30, 2017, 9:12 p.m., Nate Cole wrote: > > This is making the agent-side do the logic. Shouldn't we check these > > things and not include GPL repos in the command to the agent? > > Sid Wagle wrote: > We would still need to make sure existing hosts get the changes if user > denies license during setup and then accepts it. Present behavior is that we > do not update the repo for existing hosts. Thereby, I was ok with limiting > the changes to the agent vs server and agent. > > Nate Cole wrote: > The command repository is being sent down and processed on every command. > > Sid Wagle wrote: > Fair enough, also what about accept and then deny. Ambari should reflect > presence or absence of the repo info based on the flag. Do you think doing > this on the server would be cleaner? IMO, this is pretty custom repo handling > in any case, we do not have a repo-tag -> license-tag mapping anywhere so > wouldn't it be equally ugly? > > Sid Wagle wrote: > By license-tag I meant some lookup to figure out whether license was > accepted. > > Andrew Onischuk wrote: > IMO this looks pretty much the same whether the check is on server or > agent. However it may take some time while I re-implement this to server if > that's the approach you prefer. @Sid @Nate If you decide we need this, we can > create re-factoring jira (not for 2.6.1) and do this. > > Nate Cole wrote: > The whole point of CommandRepository was to send down to the agent ONLY > what is necessary to create the repo. Spreading logic around makes it > difficult to troubleshoot. It should either ALL be in java or ALL be in > python. @Andrew: NAte has avalid point I think we should address this in trunk to handled completely on the server side. Would you be able to open an Apache Jira for tracking this? Thanks. - Sid --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64209/#review192365 --- On Nov. 30, 2017, 1:28 p.m., Andrew Onischuk wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64209/ > --- > > (Updated Nov. 30, 2017, 1:28 p.m.) > > > Review request for Ambari, Attila Doroszlai, Myroslav Papirkovskyy, Nate > Cole, and Sid Wagle. > > > Bugs: AMBARI-22561 > https://issues.apache.org/jira/browse/AMBARI-22561 > > > Repository: ambari > > > Description > --- > > * User denies the GPL license agreement, UI will still issue PUT call to > create the repo > * Ambari should not write HDP-GPL repo info to the ambari-hdp-1.repo > because it will break yum in local repo world. > * Then the user accepts the license by running ambari-server setup > * The repo file does not get updated on existing hosts > * We need to push the HDP-GPL repo to the repo files on existing hosts only > when the license is accepted > > > Diffs > - > > > ambari-common/src/main/python/resource_management/libraries/functions/lzo_utils.py > 68ee607b9b > > ambari-common/src/main/python/resource_management/libraries/functions/repository_util.py > f1c8ef1b04 > > ambari-common/src/main/python/resource_management/libraries/script/script.py > ccb8a547f4 > > ambari-server/src/main/java/org/apache/ambari/server/agent/CommandRepository.java > e207ac5289 > ambari-server/src/main/resources/custom_actions/scripts/install_packages.py > b819dcad96 > > ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/repo_initialization.py > 82e57aa4f4 > ambari-server/src/test/python/stacks/utils/RMFTestCase.py 80712c5ebe > > > Diff: https://reviews.apache.org/r/64209/diff/1/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Andrew Onischuk > >
Re: Review Request 64221: AMBARI-22567 : Integrate Spark lifecycle management into AMS AD Manager.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64221/#review192380 --- Fix it, then Ship it! ambari-metrics/ambari-metrics-anomaly-detection-service/conf/unix/ambari-metrics-admanager Lines 33 (patched) <https://reviews.apache.org/r/64221/#comment270480> If this script retruns a ret code we should be using it here. ambari-metrics/ambari-metrics-anomaly-detection-service/conf/unix/ambari-metrics-admanager Lines 35 (patched) <https://reviews.apache.org/r/64221/#comment270481> Have another if, maybe we want to add status too if spark script supports it. ambari-metrics/ambari-metrics-anomaly-detection-service/conf/unix/log4j.properties Line 25 (original), 25 (patched) <https://reviews.apache.org/r/64221/#comment270482> Uncommon path usage ambari-metrics/ambari-metrics-assembly/pom.xml Lines 662 (patched) <https://reviews.apache.org/r/64221/#comment270483> Need to make sure we add debian support. ambari-metrics/ambari-metrics-assembly/src/main/assembly/anomaly-detection.xml Lines 21 (patched) <https://reviews.apache.org/r/64221/#comment270484> formatting: Tabs are 4 spaces instead of 2. ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-ad-spark-env.xml Lines 22 (patched) <https://reviews.apache.org/r/64221/#comment270486> formatting ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-ad-spark-env.xml Lines 24 (patched) <https://reviews.apache.org/r/64221/#comment270488> MAster should have less memory that worker by default. ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-ad-spark-env.xml Lines 72 (patched) <https://reviews.apache.org/r/64221/#comment270489> Move lower than master and worker settings for readbility. ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/metainfo.xml Lines 141 (patched) <https://reviews.apache.org/r/64221/#comment270485> ams-admanager-spark-env ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/ams.py Lines 574 (patched) <https://reviews.apache.org/r/64221/#comment270491> Better to do fixed loc from rpm. - Sid Wagle On Nov. 30, 2017, 9:34 p.m., Aravindan Vijayan wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64221/ > --- > > (Updated Nov. 30, 2017, 9:34 p.m.) > > > Review request for Ambari, Jayush Luniya and Sid Wagle. > > > Bugs: AMBARI-22567 > https://issues.apache.org/jira/browse/AMBARI-22567 > > > Repository: ambari > > > Description > --- > > Package spark tar ball along with AD Manager jar. > Implement START/STOP of embedded spark master + worker as a step in AD > Manager start/stop. > Build ambari-metrics-admanager RPM > Verify ability to configure and deploy AD Manager using Ambari. > > > Diffs > - > > > ambari-metrics/ambari-metrics-anomaly-detection-service/conf/unix/ambari-metrics-admanager > PRE-CREATION > > ambari-metrics/ambari-metrics-anomaly-detection-service/conf/unix/ambari-metrics-admanager.sh > f1a1ae3 > > ambari-metrics/ambari-metrics-anomaly-detection-service/conf/unix/config.yaml > PRE-CREATION > > ambari-metrics/ambari-metrics-anomaly-detection-service/conf/unix/log4j.properties > 9dba1da > ambari-metrics/ambari-metrics-anomaly-detection-service/pom.xml c6927dd > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/assemblies/empty.xml > PRE-CREATION > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/resources/config.yml > 7de06b4 > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/app/AnomalyDetectionAppConfig.scala > f9ed4b2 > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/common/Range.scala > 003c18f > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/common/Season.scala > f875e3b > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/common/SeasonType.scala > 067972c > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/common/TimeRange.scala > 50df658 > > ambari-metrics/ambari-metrics-anomaly-d
Re: Review Request 64209: Need to address HDP-GPL repo update after user accepts license in post-install scenario
> On Nov. 30, 2017, 9:12 p.m., Nate Cole wrote: > > This is making the agent-side do the logic. Shouldn't we check these > > things and not include GPL repos in the command to the agent? > > Sid Wagle wrote: > We would still need to make sure existing hosts get the changes if user > denies license during setup and then accepts it. Present behavior is that we > do not update the repo for existing hosts. Thereby, I was ok with limiting > the changes to the agent vs server and agent. > > Nate Cole wrote: > The command repository is being sent down and processed on every command. > > Sid Wagle wrote: > Fair enough, also what about accept and then deny. Ambari should reflect > presence or absence of the repo info based on the flag. Do you think doing > this on the server would be cleaner? IMO, this is pretty custom repo handling > in any case, we do not have a repo-tag -> license-tag mapping anywhere so > wouldn't it be equally ugly? By license-tag I meant some lookup to figure out whether license was accepted. - Sid --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64209/#review192365 --- On Nov. 30, 2017, 1:28 p.m., Andrew Onischuk wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64209/ > --- > > (Updated Nov. 30, 2017, 1:28 p.m.) > > > Review request for Ambari, Attila Doroszlai, Myroslav Papirkovskyy, Nate > Cole, and Sid Wagle. > > > Bugs: AMBARI-22561 > https://issues.apache.org/jira/browse/AMBARI-22561 > > > Repository: ambari > > > Description > --- > > * User denies the GPL license agreement, UI will still issue PUT call to > create the repo > * Ambari should not write HDP-GPL repo info to the ambari-hdp-1.repo > because it will break yum in local repo world. > * Then the user accepts the license by running ambari-server setup > * The repo file does not get updated on existing hosts > * We need to push the HDP-GPL repo to the repo files on existing hosts only > when the license is accepted > > > Diffs > - > > > ambari-common/src/main/python/resource_management/libraries/functions/lzo_utils.py > 68ee607b9b > > ambari-common/src/main/python/resource_management/libraries/functions/repository_util.py > f1c8ef1b04 > > ambari-common/src/main/python/resource_management/libraries/script/script.py > ccb8a547f4 > > ambari-server/src/main/java/org/apache/ambari/server/agent/CommandRepository.java > e207ac5289 > ambari-server/src/main/resources/custom_actions/scripts/install_packages.py > b819dcad96 > > ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/repo_initialization.py > 82e57aa4f4 > ambari-server/src/test/python/stacks/utils/RMFTestCase.py 80712c5ebe > > > Diff: https://reviews.apache.org/r/64209/diff/1/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Andrew Onischuk > >
Re: Review Request 64209: Need to address HDP-GPL repo update after user accepts license in post-install scenario
> On Nov. 30, 2017, 9:12 p.m., Nate Cole wrote: > > This is making the agent-side do the logic. Shouldn't we check these > > things and not include GPL repos in the command to the agent? > > Sid Wagle wrote: > We would still need to make sure existing hosts get the changes if user > denies license during setup and then accepts it. Present behavior is that we > do not update the repo for existing hosts. Thereby, I was ok with limiting > the changes to the agent vs server and agent. > > Nate Cole wrote: > The command repository is being sent down and processed on every command. Fair enough, also what about accept and then deny. Ambari should reflect presence or absence of the repo info based on the flag. Do you think doing this on the server would be cleaner? IMO, this is pretty custom repo handling in any case, we do not have a repo-tag -> license-tag mapping anywhere so wouldn't it be equally ugly? - Sid --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64209/#review192365 --- On Nov. 30, 2017, 1:28 p.m., Andrew Onischuk wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64209/ > --- > > (Updated Nov. 30, 2017, 1:28 p.m.) > > > Review request for Ambari, Attila Doroszlai, Myroslav Papirkovskyy, Nate > Cole, and Sid Wagle. > > > Bugs: AMBARI-22561 > https://issues.apache.org/jira/browse/AMBARI-22561 > > > Repository: ambari > > > Description > --- > > * User denies the GPL license agreement, UI will still issue PUT call to > create the repo > * Ambari should not write HDP-GPL repo info to the ambari-hdp-1.repo > because it will break yum in local repo world. > * Then the user accepts the license by running ambari-server setup > * The repo file does not get updated on existing hosts > * We need to push the HDP-GPL repo to the repo files on existing hosts only > when the license is accepted > > > Diffs > - > > > ambari-common/src/main/python/resource_management/libraries/functions/lzo_utils.py > 68ee607b9b > > ambari-common/src/main/python/resource_management/libraries/functions/repository_util.py > f1c8ef1b04 > > ambari-common/src/main/python/resource_management/libraries/script/script.py > ccb8a547f4 > > ambari-server/src/main/java/org/apache/ambari/server/agent/CommandRepository.java > e207ac5289 > ambari-server/src/main/resources/custom_actions/scripts/install_packages.py > b819dcad96 > > ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/repo_initialization.py > 82e57aa4f4 > ambari-server/src/test/python/stacks/utils/RMFTestCase.py 80712c5ebe > > > Diff: https://reviews.apache.org/r/64209/diff/1/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Andrew Onischuk > >
Re: Review Request 64209: Need to address HDP-GPL repo update after user accepts license in post-install scenario
> On Nov. 30, 2017, 9:12 p.m., Nate Cole wrote: > > This is making the agent-side do the logic. Shouldn't we check these > > things and not include GPL repos in the command to the agent? We would still need to make sure existing hosts get the changes if user denies license during setup and then accepts it. Present behavior is that we do not update the repo for existing hosts. Thereby, I was ok with limiting the changes to the agent vs server and agent. - Sid --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64209/#review192365 --- On Nov. 30, 2017, 1:28 p.m., Andrew Onischuk wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64209/ > --- > > (Updated Nov. 30, 2017, 1:28 p.m.) > > > Review request for Ambari, Attila Doroszlai, Myroslav Papirkovskyy, Nate > Cole, and Sid Wagle. > > > Bugs: AMBARI-22561 > https://issues.apache.org/jira/browse/AMBARI-22561 > > > Repository: ambari > > > Description > --- > > * User denies the GPL license agreement, UI will still issue PUT call to > create the repo > * Ambari should not write HDP-GPL repo info to the ambari-hdp-1.repo > because it will break yum in local repo world. > * Then the user accepts the license by running ambari-server setup > * The repo file does not get updated on existing hosts > * We need to push the HDP-GPL repo to the repo files on existing hosts only > when the license is accepted > > > Diffs > - > > > ambari-common/src/main/python/resource_management/libraries/functions/lzo_utils.py > 68ee607b9b > > ambari-common/src/main/python/resource_management/libraries/functions/repository_util.py > f1c8ef1b04 > > ambari-common/src/main/python/resource_management/libraries/script/script.py > ccb8a547f4 > > ambari-server/src/main/java/org/apache/ambari/server/agent/CommandRepository.java > e207ac5289 > ambari-server/src/main/resources/custom_actions/scripts/install_packages.py > b819dcad96 > > ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/repo_initialization.py > 82e57aa4f4 > ambari-server/src/test/python/stacks/utils/RMFTestCase.py 80712c5ebe > > > Diff: https://reviews.apache.org/r/64209/diff/1/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Andrew Onischuk > >
Re: Review Request 64209: Need to address HDP-GPL repo update after user accepts license in post-install scenario
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64209/#review192335 --- Ship it! Can you provide some notes on the manual testing effort on this? - Sid Wagle On Nov. 30, 2017, 1:28 p.m., Andrew Onischuk wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64209/ > --- > > (Updated Nov. 30, 2017, 1:28 p.m.) > > > Review request for Ambari, Attila Doroszlai, Myroslav Papirkovskyy, Nate > Cole, and Sid Wagle. > > > Bugs: AMBARI-22561 > https://issues.apache.org/jira/browse/AMBARI-22561 > > > Repository: ambari > > > Description > --- > > * User denies the GPL license agreement, UI will still issue PUT call to > create the repo > * Ambari should not write HDP-GPL repo info to the ambari-hdp-1.repo > because it will break yum in local repo world. > * Then the user accepts the license by running ambari-server setup > * The repo file does not get updated on existing hosts > * We need to push the HDP-GPL repo to the repo files on existing hosts only > when the license is accepted > > > Diffs > - > > > ambari-common/src/main/python/resource_management/libraries/functions/lzo_utils.py > 68ee607b9b > > ambari-common/src/main/python/resource_management/libraries/functions/repository_util.py > f1c8ef1b04 > > ambari-common/src/main/python/resource_management/libraries/script/script.py > ccb8a547f4 > > ambari-server/src/main/java/org/apache/ambari/server/agent/CommandRepository.java > e207ac5289 > ambari-server/src/main/resources/custom_actions/scripts/install_packages.py > b819dcad96 > > ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/repo_initialization.py > 82e57aa4f4 > ambari-server/src/test/python/stacks/utils/RMFTestCase.py 80712c5ebe > > > Diff: https://reviews.apache.org/r/64209/diff/1/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Andrew Onischuk > >
Re: Review Request 64186: Update server setup to deny GPL as default
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64186/ --- (Updated Nov. 30, 2017, 6:36 p.m.) Review request for Ambari, Andrew Onischuk, Jonathan Hurley, and Myroslav Papirkovskyy. Changes --- Minor change to write out false on silent without accept. Bugs: AMBARI-22555 https://issues.apache.org/jira/browse/AMBARI-22555 Repository: ambari Description --- Presently the default is to accept GPL for LZO download. Diffs - ambari-server/src/main/python/ambari_server/serverConfiguration.py 8697e4c ambari-server/src/main/python/ambari_server/serverUpgrade.py a864609 ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 3f8a772 ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py e5adb19 Diff: https://reviews.apache.org/r/64186/diff/3/ Testing --- mvn clean test File Attachments (updated) AMBARI-22555-1.patch https://reviews.apache.org/media/uploaded/files/2017/11/30/3aaac156-4020-4d03-8611-de2790ff9301__AMBARI-22555-1.patch AMBARI-22555-1.patch https://reviews.apache.org/media/uploaded/files/2017/11/30/1276fb76-4340-4d2b-8bb2-ff2edfa49a73__AMBARI-22555-1.patch Thanks, Sid Wagle
Re: Review Request 64186: Update server setup to deny GPL as default
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64186/ --- (Updated Nov. 30, 2017, 6:26 p.m.) Review request for Ambari, Andrew Onischuk, Jonathan Hurley, and Myroslav Papirkovskyy. Changes --- Fix for silent accept. Bugs: AMBARI-22555 https://issues.apache.org/jira/browse/AMBARI-22555 Repository: ambari Description --- Presently the default is to accept GPL for LZO download. Diffs - ambari-server/src/main/python/ambari_server/serverConfiguration.py 8697e4c ambari-server/src/main/python/ambari_server/serverUpgrade.py a864609 ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 3f8a772 ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py e5adb19 Diff: https://reviews.apache.org/r/64186/diff/3/ Testing --- mvn clean test File Attachments (updated) AMBARI-22555-1.patch https://reviews.apache.org/media/uploaded/files/2017/11/30/3aaac156-4020-4d03-8611-de2790ff9301__AMBARI-22555-1.patch Thanks, Sid Wagle
Re: Review Request 64174: Upgrade from Ambari 2.5.2 to 2.6.0 fails.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64174/#review192315 --- Ship it! Ship It! - Sid Wagle On Nov. 29, 2017, 8:49 p.m., Myroslav Papirkovskyy wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64174/ > --- > > (Updated Nov. 29, 2017, 8:49 p.m.) > > > Review request for Ambari, Andrew Onischuk and Sid Wagle. > > > Bugs: AMBARI-22544 > https://issues.apache.org/jira/browse/AMBARI-22544 > > > Repository: ambari > > > Description > --- > > When upgrading from Ambari 2.5.2 to Ambari 2.6.0, we hit this error > preventing from completing the ambari-server upgrade. > > ``` > 20 Nov 2017 11:39:08,115 ERROR [main] SchemaUpgradeHelper:202 - Upgrade > failed. > com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: > Cannot delete or update a parent row: a foreign key constraint fails > (`ambari_01`.`clusters`, CONSTRAINT `FK_clusters_upgrade_id` FOREIGN KEY > (`upgrade_id`) REFERENCES `upgrade` (`upgrade_id`)) > ``` > > To get around this, we had to restore the db backup from 2.5.3 and adjust a > table in mysql with ‘update clusters set upgrade_id = null’ so a record in > the ‘upgrades’ table could be removed. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog260.java > 35f9f65dc89302aaf4c555d269beb221f2c0a793 > > ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog260Test.java > c6111712c3f79347234d282461e0d9f0ed5a8b75 > > > Diff: https://reviews.apache.org/r/64174/diff/1/ > > > Testing > --- > > mvn clean test > manual check of described case > > > Thanks, > > Myroslav Papirkovskyy > >
Re: Review Request 64216: . Repository validation fails for HDP-GPL
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64216/#review192314 --- Ship it! Ship It! - Sid Wagle On Nov. 30, 2017, 5:14 p.m., Myroslav Papirkovskyy wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64216/ > --- > > (Updated Nov. 30, 2017, 5:14 p.m.) > > > Review request for Ambari, Andrew Onischuk, Jonathan Hurley, and Sid Wagle. > > > Bugs: AMBARI-22565 > https://issues.apache.org/jira/browse/AMBARI-22565 > > > Repository: ambari > > > Description > --- > > Server always fail to validate repository with unknown repo_id. > We perform repository lookup in stack only to get repo_name. Should use > repo_name received from API request. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java > 48ab1b93e880e6e46c3e6f70dc433a5a3643a2ee > > ambari-server/src/main/java/org/apache/ambari/server/controller/RepositoryRequest.java > 84929bfe0eb498917401445b5ce8cd8a015bfa0f > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RepositoryResourceProvider.java > cec43dd40f20061965cc98cc357d193a76a16533 > > ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerImplTest.java > 00b87a69224c35a25c3c0e0ec39a9bbd4fc7a9b8 > > ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerTest.java > 6e9e5efcd408b683224db46e668a44336a9cf167 > > > Diff: https://reviews.apache.org/r/64216/diff/1/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Myroslav Papirkovskyy > >
Re: Review Request 64186: Update server setup to deny GPL as default
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64186/ --- (Updated Nov. 30, 2017, 12:20 a.m.) Review request for Ambari, Andrew Onischuk, Jonathan Hurley, and Myroslav Papirkovskyy. Changes --- Fixed typo Bugs: AMBARI-22555 https://issues.apache.org/jira/browse/AMBARI-22555 Repository: ambari Description --- Presently the default is to accept GPL for LZO download. Diffs (updated) - ambari-server/src/main/python/ambari_server/serverConfiguration.py 8697e4c ambari-server/src/main/python/ambari_server/serverUpgrade.py a864609 ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 3f8a772 ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py e5adb19 Diff: https://reviews.apache.org/r/64186/diff/3/ Changes: https://reviews.apache.org/r/64186/diff/2-3/ Testing --- mvn clean test Thanks, Sid Wagle
Re: Review Request 64186: Update server setup to deny GPL as default
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64186/ --- (Updated Nov. 30, 2017, 12:02 a.m.) Review request for Ambari, Andrew Onischuk, Jonathan Hurley, and Myroslav Papirkovskyy. Bugs: AMBARI-22555 https://issues.apache.org/jira/browse/AMBARI-22555 Repository: ambari Description --- Presently the default is to accept GPL for LZO download. Diffs (updated) - ambari-server/src/main/python/ambari_server/serverConfiguration.py 8697e4c ambari-server/src/main/python/ambari_server/serverUpgrade.py a864609 ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 3f8a772 ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py e5adb19 Diff: https://reviews.apache.org/r/64186/diff/2/ Changes: https://reviews.apache.org/r/64186/diff/1-2/ Testing (updated) --- mvn clean test Thanks, Sid Wagle
Review Request 64186: Update server setup to deny GPL as default
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64186/ --- Review request for Ambari, Andrew Onischuk, Jonathan Hurley, and Myroslav Papirkovskyy. Bugs: AMBARI-22555 https://issues.apache.org/jira/browse/AMBARI-22555 Repository: ambari Description --- Presently the default is to accept GPL for LZO download. Diffs - ambari-server/src/main/python/ambari_server/serverConfiguration.py 8697e4c ambari-server/src/main/python/ambari_server/serverUpgrade.py a864609 ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 3f8a772 Diff: https://reviews.apache.org/r/64186/diff/1/ Testing --- Thanks, Sid Wagle
Re: Review Request 64174: Upgrade from Ambari 2.5.2 to 2.6.0 fails.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64174/#review192169 --- ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog260.java Lines 350 (patched) <https://reviews.apache.org/r/64174/#comment270214> Will this delete existing data? Any implications of doing so? - Sid Wagle On Nov. 29, 2017, 8:49 p.m., Myroslav Papirkovskyy wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64174/ > --- > > (Updated Nov. 29, 2017, 8:49 p.m.) > > > Review request for Ambari, Andrew Onischuk and Sid Wagle. > > > Bugs: AMBARI-22544 > https://issues.apache.org/jira/browse/AMBARI-22544 > > > Repository: ambari > > > Description > --- > > When upgrading from Ambari 2.5.2 to Ambari 2.6.0, we hit this error > preventing from completing the ambari-server upgrade. > > ``` > 20 Nov 2017 11:39:08,115 ERROR [main] SchemaUpgradeHelper:202 - Upgrade > failed. > com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: > Cannot delete or update a parent row: a foreign key constraint fails > (`ambari_01`.`clusters`, CONSTRAINT `FK_clusters_upgrade_id` FOREIGN KEY > (`upgrade_id`) REFERENCES `upgrade` (`upgrade_id`)) > ``` > > To get around this, we had to restore the db backup from 2.5.3 and adjust a > table in mysql with ‘update clusters set upgrade_id = null’ so a record in > the ‘upgrades’ table could be removed. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog260.java > 35f9f65dc89302aaf4c555d269beb221f2c0a793 > > ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog260Test.java > c6111712c3f79347234d282461e0d9f0ed5a8b75 > > > Diff: https://reviews.apache.org/r/64174/diff/1/ > > > Testing > --- > > mvn clean test > manual check of described case > > > Thanks, > > Myroslav Papirkovskyy > >
Re: Review Request 64134: Remove Deprecated Non-Versioned LZO Packages from LZO Install
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64134/#review192066
---
Ship it!
Ship It!
- Sid Wagle
On Nov. 28, 2017, 9:24 p.m., Jonathan Hurley wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64134/
> ---
>
> (Updated Nov. 28, 2017, 9:24 p.m.)
>
>
> Review request for Ambari, Nate Cole and Sid Wagle.
>
>
> Bugs: AMBARI-22536
> https://issues.apache.org/jira/browse/AMBARI-22536
>
>
> Repository: ambari
>
>
> Description
> ---
>
> Ambari was previously installed 4 LZO packages:
>
> - lzo / liblzo2-2
> - hadoop-lzo
> - hadoop-lzo-native
> - hadooplzo_${stack_version}
> - hadooplzo_${stack_version}-native
>
> The non-versioned packages contain the same files and should not be used
> anymore:
>
> ```
> /usr/hdp/current/share/lzo/0.6.0/lib/hadoop-lzo-0.6.0.jar
>
> [root@c6401 usr]# repoquery -l hadooplzo_2_6_3_0_230.x86_64
> /usr/hdp/2.6.3.0-230/hadoop/lib/hadoop-lzo-0.6.0.2.6.3.0-230-javadoc.jar
> /usr/hdp/2.6.3.0-230/hadoop/lib/hadoop-lzo-0.6.0.2.6.3.0-230-sources.jar
> /usr/hdp/2.6.3.0-230/hadoop/lib/hadoop-lzo-0.6.0.2.6.3.0-230.jar
> ```
>
> ```
> [root@c6401 share]# repoquery -l hadoop-lzo-native
> /usr/hdp/current/share/lzo/0.6.0/lib/native
> /usr/hdp/current/share/lzo/0.6.0/lib/native/Linux-amd64-64
> /usr/hdp/current/share/lzo/0.6.0/lib/native/Linux-amd64-64/libgplcompression.a
> /usr/hdp/current/share/lzo/0.6.0/lib/native/Linux-amd64-64/libgplcompression.la
> /usr/hdp/current/share/lzo/0.6.0/lib/native/Linux-amd64-64/libgplcompression.so
> /usr/hdp/current/share/lzo/0.6.0/lib/native/Linux-amd64-64/libgplcompression.so.0
> /usr/hdp/current/share/lzo/0.6.0/lib/native/Linux-amd64-64/libgplcompression.so.0.0.0
> /usr/hdp/current/share/lzo/0.6.0/lib/native/docs
>
> [root@c6401 share]# repoquery -l hadooplzo_2_6_3_0_230-native.x86_64
> /usr/hdp/2.6.3.0-230/hadoop/lib/native
> /usr/hdp/2.6.3.0-230/hadoop/lib/native/Linux-amd64-64
> /usr/hdp/2.6.3.0-230/hadoop/lib/native/Linux-amd64-64/libgplcompression.a
> /usr/hdp/2.6.3.0-230/hadoop/lib/native/Linux-amd64-64/libgplcompression.la
> /usr/hdp/2.6.3.0-230/hadoop/lib/native/Linux-amd64-64/libgplcompression.so
> /usr/hdp/2.6.3.0-230/hadoop/lib/native/Linux-amd64-64/libgplcompression.so.0
> /usr/hdp/2.6.3.0-230/hadoop/lib/native/Linux-amd64-64/libgplcompression.so.0.0.0
> ```
>
>
> Diffs
> -
>
>
> ambari-common/src/main/python/resource_management/libraries/functions/lzo_utils.py
> d6d987fda9
>
>
> Diff: https://reviews.apache.org/r/64134/diff/1/
>
>
> Testing
> ---
>
> --
> Total run:1196
> Total errors:0
> Total failures:0
> OK
> [INFO]
>
> [INFO] BUILD SUCCESS
> [INFO]
>
> [INFO] Total time: 57.608 s
> [INFO] Finished at: 2017-11-28T16:23:16-05:00
> [INFO] Final Memory: 18M/491M
> [INFO]
>
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 64127: Remove extjs From Oozie Installations By Default
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64127/#review192044
---
Ship it!
Ship It!
- Sid Wagle
On Nov. 28, 2017, 6:54 p.m., Jonathan Hurley wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64127/
> ---
>
> (Updated Nov. 28, 2017, 6:54 p.m.)
>
>
> Review request for Ambari, Nate Cole and Sid Wagle.
>
>
> Bugs: AMBARI-22535
> https://issues.apache.org/jira/browse/AMBARI-22535
>
>
> Repository: ambari
>
>
> Description
> ---
>
> GPL code was recently removed from some repositories, such as HDP-UTILS.
> Oozie uses a GPL module called extJS which is installed by default with no
> option not to install it.
>
> {code}
> 2017-11-28 17:00:17,813 - Execution of '/usr/bin/yum -d 0 -e 0 -y install
> extjs' returned 1. Error: Nothing to do
> 2017-11-28 17:00:17,814 - Failed to install package extjs. Executing
> '/usr/bin/yum clean metadata'
> 2017-11-28 17:00:18,075 - Retrying to install package extjs after 30 seconds
> {code}
>
> Since it no longer is included in the repos, it should be removed. Oozie can
> still load and start without it, but the web client will not be available.
>
> **I could not see a way to remove a dependency we need to keep for HDP 2.5
> without re-pasting the entire osSpecifics section back into HDP 2.6 with the
> extjs stuff cut out**
>
>
> Diffs
> -
>
> ambari-server/src/main/resources/stacks/HDP/2.6/services/OOZIE/metainfo.xml
> c9dfaadf37
>
>
> Diff: https://reviews.apache.org/r/64127/diff/1/
>
>
> Testing
> ---
>
> Installed Oozie on HDP 2.6 and verified SC's passed and the UI was returning
> an HTTP 200 with a page saying the UI was disabled.
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 64082: PERF stack: deploy fails due to incorrect stack upgrade related information
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64082/#review191904 --- Ship it! Ship It! - Sid Wagle On Nov. 27, 2017, 1:17 p.m., Andrew Onischuk wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64082/ > --- > > (Updated Nov. 27, 2017, 1:17 p.m.) > > > Review request for Ambari, Attila Doroszlai, Sid Wagle, and Vitalyi > Brodetskyi. > > > Bugs: AMBARI-22520 > https://issues.apache.org/jira/browse/AMBARI-22520 > > > Repository: ambari > > > Description > --- > > > Diffs > - > > > ambari-server/src/main/resources/stacks/PERF/1.0/configuration/cluster-env.xml > f19ac5237e > > ambari-server/src/main/resources/stacks/PERF/1.0/properties/stack_packages.json > PRE-CREATION > > > Diff: https://reviews.apache.org/r/64082/diff/1/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Andrew Onischuk > >
Re: Review Request 64087: Service config changes on hdp + hdf cluster throws Consistency check failed
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64087/#review191903 --- Ship it! Ship It! - Sid Wagle On Nov. 27, 2017, 4:01 p.m., Myroslav Papirkovskyy wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64087/ > --- > > (Updated Nov. 27, 2017, 4:01 p.m.) > > > Review request for Ambari, Andrew Onischuk and Sid Wagle. > > > Bugs: AMBARI-22523 > https://issues.apache.org/jira/browse/AMBARI-22523 > > > Repository: ambari > > > Description > --- > > Consistency check fails with unknown error on config change if HIVE and DRUID > services are present. > > > Diffs > - > > ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py > a5f54a346379a8fa4252184321620557a06344d3 > ambari-server/src/test/python/stacks/2.6/common/test_stack_advisor.py > f4c550897d75d54d52810f3070cfc0e7c8742173 > > > Diff: https://reviews.apache.org/r/64087/diff/1/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Myroslav Papirkovskyy > >
Re: Review Request 63957: Stack advisor should disallow lzo enable without accepting license agreement.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63957/#review191752 --- Ship it! Ship It! - Sid Wagle On Nov. 22, 2017, 6:09 p.m., Myroslav Papirkovskyy wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63957/ > --- > > (Updated Nov. 22, 2017, 6:09 p.m.) > > > Review request for Ambari, Andrew Onischuk, Jonathan Hurley, Nate Cole, and > Sid Wagle. > > > Bugs: AMBARI-22484 > https://issues.apache.org/jira/browse/AMBARI-22484 > > > Repository: ambari > > > Description > --- > > If the user adds com.hadoop.compression.lzo.LzoCodec to core-site property > io.compression.codec.lzo.class and the ambari.properties does not have the > allow GPL software install flag, throw validation error > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/StackAdvisorRequest.java > 64180e5039bdb74d3aaf25e140565c82a95a26c1 > > ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/commands/StackAdvisorCommand.java > 5440462b86cdcfafb1e9ab239dacfddee0b31b93 > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java > b2a68d66d777f107fdc4f5576a691e8d25ec2c92 > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackAdvisorResourceProvider.java > dd32c6badb18c1a8c02b7093813cd92ccd2fc74c > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py > c36a41f49b224a904368c7474f0f769348e65ae6 > ambari-server/src/main/resources/stacks/stack_advisor.py > 8ccbaba9db083068268951ae4e07ce08d32edcc4 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ValidationResourceProviderTest.java > cdbf2d782f86b2a30f54619c4e2a9165a020275c > ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py > 9e0070458987ee490f3918fead5ef11d201c8612 > > > Diff: https://reviews.apache.org/r/63957/diff/2/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Myroslav Papirkovskyy > >
Re: Review Request 63957: Stack advisor should disallow lzo enable without accepting license agreement.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63957/#review191533 --- Ship it! Ship It! - Sid Wagle On Nov. 20, 2017, 5:06 p.m., Myroslav Papirkovskyy wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63957/ > --- > > (Updated Nov. 20, 2017, 5:06 p.m.) > > > Review request for Ambari, Andrew Onischuk, Jonathan Hurley, Nate Cole, and > Sid Wagle. > > > Bugs: AMBARI-22484 > https://issues.apache.org/jira/browse/AMBARI-22484 > > > Repository: ambari > > > Description > --- > > If the user adds com.hadoop.compression.lzo.LzoCodec to core-site property > io.compression.codec.lzo.class and the ambari.properties does not have the > allow GPL software install flag, throw validation error > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/StackAdvisorRequest.java > 64180e5039bdb74d3aaf25e140565c82a95a26c1 > > ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/commands/StackAdvisorCommand.java > 5440462b86cdcfafb1e9ab239dacfddee0b31b93 > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java > b2a68d66d777f107fdc4f5576a691e8d25ec2c92 > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackAdvisorResourceProvider.java > dd32c6badb18c1a8c02b7093813cd92ccd2fc74c > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py > c36a41f49b224a904368c7474f0f769348e65ae6 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ValidationResourceProviderTest.java > cdbf2d782f86b2a30f54619c4e2a9165a020275c > ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py > 9e0070458987ee490f3918fead5ef11d201c8612 > > > Diff: https://reviews.apache.org/r/63957/diff/1/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Myroslav Papirkovskyy > >
Re: Review Request 63935: AMBARI-22470 : Refine Metric Definition Service and AD Query service.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63935/#review191437 --- Fix it, then Ship it! ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/db/DefaultPhoenixDataSource.scala Lines 1 (patched) <https://reviews.apache.org/r/63935/#comment269251> 2 notice copies. ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/db/DefaultPhoenixDataSource.scala Lines 60 (patched) <https://reviews.apache.org/r/63935/#comment269252> Formatting, too long. ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/metadata/ADMetadataProvider.scala Lines 105 (patched) <https://reviews.apache.org/r/63935/#comment269253> Can reduce code duplication with a helper method. - Sid Wagle On Nov. 18, 2017, 4:24 a.m., Aravindan Vijayan wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63935/ > --- > > (Updated Nov. 18, 2017, 4:24 a.m.) > > > Review request for Ambari, Jayush Luniya and Sid Wagle. > > > Bugs: AMBARI-22470 > https://issues.apache.org/jira/browse/AMBARI-22470 > > > Repository: ambari > > > Description > --- > > Metric definition service - integration with LevelDB. > Metric definition service - integration with Metrics Collector > AD Query Service - Ability to retrieve anomaly data from Phoenix. > Fix pom.xml issues. > Startup metric definition and query services during server start. > Remove phoenix connection code from ambari-metrics-common collector and AD > service modules. > > > Diffs > - > > ambari-metrics/ambari-metrics-anomaly-detection-service/pom.xml 142f02f > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/resources/config.yml > 9402f6e > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/resources/hbase-site.xml > PRE-CREATION > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/app/ADServiceScalaModule.scala > PRE-CREATION > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/app/AnomalyDetectionApp.scala > 8b3a829 > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/app/AnomalyDetectionAppConfig.scala > 93f6b28 > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/app/AnomalyDetectionAppModule.scala > a896563 > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/configuration/HBaseConfiguration.scala > a51a959 > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/db/AdAnomalyStoreAccessor.scala > PRE-CREATION > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/db/AdMetadataStoreAccessorImpl.scala > PRE-CREATION > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/db/ConnectionProvider.scala > PRE-CREATION > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/db/DefaultPhoenixDataSource.scala > PRE-CREATION > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/db/LevelDbStoreAccessor.scala > baad57d > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/db/MetadataDatasource.scala > aa6694a > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/db/PhoenixAnomalyStoreAccessor.scala > 36aea21 > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/db/PhoenixConnectionProvider.scala > PRE-CREATION > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/db/PhoenixQueryConstants.scala > 5379c91 > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/leveldb/LevelDBDatasource.scala > a34a60a > > ambari-metrics/ambari-metrics-anomaly-
Re: Review Request 63876: ambari-server upgrade to 2.6.1 should surface the GPL agreement
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63876/#review191210 --- Fix it, then Ship it! ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog261.java Lines 32 (patched) <https://reviews.apache.org/r/63876/#comment268862> This import ordering might not conform to those enforced by ambari-utility, make sure to address it in your IDE. ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog261.java Lines 90 (patched) <https://reviews.apache.org/r/63876/#comment268870> To make this future proof we should have a cluter map returned but for now lets add a TODO for it. - Sid Wagle On Nov. 16, 2017, 1:27 p.m., Andrew Onischuk wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63876/ > --- > > (Updated Nov. 16, 2017, 1:27 p.m.) > > > Review request for Ambari, Attila Doroszlai, Myroslav Papirkovskyy, and Sid > Wagle. > > > Bugs: AMBARI-22454 > https://issues.apache.org/jira/browse/AMBARI-22454 > > > Repository: ambari > > > Description > --- > > * ambari-server upgrade should determine whether lzo is enabled. > * If yes: Surface the same agreement as the setup step > * If no: Provide link to the documentation to disable lzo configuration > and cleanup local repositories. > > Since this involves reading from the database we should think of sending back > data on the pipe as stdout from the upgrade jvm and the python process. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java > dac871ea28 > > ambari-server/src/main/java/org/apache/ambari/server/upgrade/SchemaUpgradeHelper.java > a2dea40f58 > > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog.java > 21273fd49a > > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog261.java > PRE-CREATION > ambari-server/src/main/python/ambari_server/serverConfiguration.py > 262e9447e5 > ambari-server/src/main/python/ambari_server/serverSetup.py 8a14066a66 > ambari-server/src/main/python/ambari_server/serverUpgrade.py 421adfc4a1 > > > Diff: https://reviews.apache.org/r/63876/diff/1/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Andrew Onischuk > >
Re: Review Request 63838: ambari-server setup should surface GPL software agreement
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63838/#review191077 --- Fix it, then Ship it! ambari-server/src/main/python/ambari-server.py Lines 516 (patched) <https://reviews.apache.org/r/63838/#comment268670> Rename to be more explicit based o nthe realtime board: --enable-lzo-under-gpl-license - Sid Wagle On Nov. 15, 2017, 2:34 p.m., Andrew Onischuk wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63838/ > --- > > (Updated Nov. 15, 2017, 2:34 p.m.) > > > Review request for Ambari, Myroslav Papirkovskyy and Sid Wagle. > > > Bugs: AMBARI-22453 > https://issues.apache.org/jira/browse/AMBARI-22453 > > > Repository: ambari > > > Description > --- > > * Ask the user if they are okay with installing GPL license software > * Surface the license agreement subtext, for now use placeholder > * Save the result in ambari.properties > * The user selection should be available through API for the client and > also stack scripts > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/agent/ExecutionCommand.java > 91df506478 > > ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java > 0bd6af6611 > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java > d8712f2c75 > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClientConfigResourceProvider.java > 0b048ad089 > ambari-server/src/main/python/ambari-server.py 60c60052cc > ambari-server/src/main/python/ambari_server/serverConfiguration.py > c7ac13476f > ambari-server/src/main/python/ambari_server/serverSetup.py c1643f2621 > > > Diff: https://reviews.apache.org/r/63838/diff/1/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Andrew Onischuk > >
Re: Review Request 63787: AMBARI-22437 : Create an 'AD Manager' component in Ambari Metrics Service stack side.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63787/#review190921 --- Fix it, then Ship it! ambari-metrics/ambari-metrics-anomaly-detection-service/conf/unix/log4j.properties Lines 25 (patched) <https://reviews.apache.org/r/63787/#comment268488> This can be valid default: /var/log/ambari-metrics-anomaly-detection/ambari-metrics-admanager.log ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/leveldb/LevelDBDatasource.scala Line 44 (original), 55 (patched) <https://reviews.apache.org/r/63787/#comment268489> why get.. vs verb for behavior spec? - Sid Wagle On Nov. 14, 2017, 6:16 a.m., Aravindan Vijayan wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63787/ > --- > > (Updated Nov. 14, 2017, 6:16 a.m.) > > > Review request for Ambari, Jayush Luniya and Sid Wagle. > > > Bugs: AMBARI-22437 > https://issues.apache.org/jira/browse/AMBARI-22437 > > > Repository: ambari > > > Description > --- > > Goals > Add a component called Anomaly-Detector in Ambari Metrics Service. > Create scripts to start/stop the component > Configuration management for Anomaly-Detector component. > Create Ambari stack scripts to manage lifecycle of the component - install, > start, stop. > > > Diffs > - > > > ambari-metrics/ambari-metrics-anomaly-detection-service/conf/unix/ambari-metrics-admanager.sh > PRE-CREATION > > ambari-metrics/ambari-metrics-anomaly-detection-service/conf/unix/log4j.properties > PRE-CREATION > ambari-metrics/ambari-metrics-anomaly-detection-service/pom.xml cfa8124 > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/resources/config.yml > 299a472 > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/app/AnomalyDetectionAppConfig.scala > aa20223 > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/app/AnomalyDetectionAppModule.scala > 28b2880 > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/configuration/HBaseConfiguration.scala > a7bbc66 > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/configuration/MetricCollectorConfiguration.scala > 9418897 > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/configuration/MetricDefinitionDBConfiguration.scala > 79a350c > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/db/LevelDbStoreAccessor.scala > PRE-CREATION > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/leveldb/LevelDBDatasource.scala > 6d185bf > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/metadata/ADMetadataProvider.scala > 3bcf4b0 > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/metadata/MetricDefinitionServiceImpl.scala > ffa9944 > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/resource/AnomalyResource.scala > c941ac3 > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/resource/MetricDefinitionResource.scala > aacea79 > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/subsystem/trend/TrendAnomalyInstance.scala > 125da34 > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/test/scala/org/apache/ambari/metrics/adservice/app/AnomalyDetectionAppConfigTest.scala > 104ccea > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/test/scala/org/apache/ambari/metrics/adservice/app/DefaultADResourceSpecTest.scala > 65cf609 > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/test/scala/org/apache/ambari/metrics/adservice/leveldb/LevelDBDataSourceTest.scala > 2ddb7b8 > > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-admanager-config.xml > 489850f > > ambari-server/src/main/resources/common-services/AMBARI_MET
Re: Review Request 63755: Check if configurations used for commands are of correct version
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63755/#review190847 --- ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py Lines 466 (patched) <https://reviews.apache.org/r/63755/#comment268422> Shouldn't this exception be caught and subsequently result in refresh of config cache instead of failing? - Sid Wagle On Nov. 13, 2017, 2:45 p.m., Andrew Onischuk wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63755/ > --- > > (Updated Nov. 13, 2017, 2:45 p.m.) > > > Review request for Ambari, Attila Doroszlai, Myroslav Papirkovskyy, and Sid > Wagle. > > > Bugs: AMBARI-22427 > https://issues.apache.org/jira/browse/AMBARI-22427 > > > Repository: ambari > > > Description > --- > > This was not reproduced, but theoretically race condition might occur causing > configuration update event to be received later than the actual commands which > needs those configs. > > > Diffs > - > > ambari-agent/src/main/python/ambari_agent/ConfigurationBuilder.py > f8bdb4299c > ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py > 0debb1b603 > > ambari-agent/src/main/python/ambari_agent/listeners/CommandsEventListener.py > ae8d400d35 > > ambari-agent/src/main/python/ambari_agent/listeners/ConfigurationEventListener.py > a4884d884e > ambari-agent/src/test/python/ambari_agent/TestAgentStompResponses.py > 162a809933 > > ambari-agent/src/test/python/ambari_agent/dummy_files/stomp/configurations_update.json > e8e1ab546d > > ambari-agent/src/test/python/ambari_agent/dummy_files/stomp/execution_commands.json > 76dac1b975 > > ambari-server/src/main/java/org/apache/ambari/server/agent/stomp/AgentClusterDataHolder.java > f9663860ee > > ambari-server/src/main/java/org/apache/ambari/server/agent/stomp/AgentConfigsHolder.java > 50779ffad4 > > ambari-server/src/main/java/org/apache/ambari/server/agent/stomp/AgentDataHolder.java > a8b0a32a30 > > ambari-server/src/main/java/org/apache/ambari/server/agent/stomp/AgentHostDataHolder.java > 7c540f99a0 > > ambari-server/src/main/java/org/apache/ambari/server/events/AgentConfigsUpdateEvent.java > 89fc8bc42a > > ambari-server/src/main/java/org/apache/ambari/server/events/ExecutionCommandEvent.java > c97ed6068a > > ambari-server/src/main/java/org/apache/ambari/server/events/publishers/AgentCommandsPublisher.java > bacdc2d896 > > > Diff: https://reviews.apache.org/r/63755/diff/2/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Andrew Onischuk > >
Re: Review Request 63671: Remove Auto-Installation of LZO Libraries
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63671/#review190474 --- Ship it! Ship It! - Sid Wagle On Nov. 8, 2017, 5:08 p.m., Nate Cole wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63671/ > --- > > (Updated Nov. 8, 2017, 5:08 p.m.) > > > Review request for Ambari, Dmitro Lisnichenko, Jonathan Hurley, and Sid Wagle. > > > Bugs: AMBARI-22383 > https://issues.apache.org/jira/browse/AMBARI-22383 > > > Repository: ambari > > > Description > --- > > The LZO library licenses are not compatible with Ambari's license and should > not be assumed to be available in Apache-based repositories. > > > Diffs > - > > > ambari-common/src/main/python/resource_management/libraries/functions/__init__.py > f144b2da32 > > ambari-common/src/main/python/resource_management/libraries/functions/get_lzo_packages.py > cfbb7d81e8 > > ambari-common/src/main/python/resource_management/libraries/functions/package_conditions.py > 31e78b9e60 > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml > 2df1b9f647 > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/datanode.py > 8bfa7e63e9 > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs.py > 6f702d3616 > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_client.py > 9c4f9d4493 > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/install_params.py > fe488c3004 > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode.py > 25534e9ff5 > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py > 89719cff2c > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py > 5ca012b796 > > ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie.py > 7d22603184 > > ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server_upgrade.py > eb57c22834 > > ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/params_linux.py > cd1f7ad36f > > ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/tez_client.py > ed3f5fd387 > ambari-server/src/main/resources/custom_actions/scripts/remove_bits.py > 0be9a34ab0 > > ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HDFS/metainfo.xml > 8500839692 > > ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HDFS/package/scripts/params.py > 3a711c721a > > ambari-server/src/main/resources/stacks/BigInsights/4.0/services/HDFS/metainfo.xml > 918cdb32fa > > ambari-server/src/main/resources/stacks/BigInsights/4.0/services/HDFS/package/scripts/params.py > 1f76f8033a > > ambari-server/src/main/resources/stacks/BigInsights/4.0/services/OOZIE/package/scripts/oozie.py > 6cd2b71f8e > > ambari-server/src/main/resources/stacks/BigInsights/4.0/services/OOZIE/package/scripts/params.py > febd3e6580 > > ambari-server/src/main/resources/stacks/BigInsights/4.1/services/HDFS/metainfo.xml > 4c00b4f3f8 > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/metainfo.xml > fc7c324ff6 > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/scripts/hdfs.py > 1264284cd4 > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/scripts/install_params.py > da8de46165 > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/scripts/params_linux.py > a77ec82cdc > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/oozie.py > 33a309fa2e > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/params_linux.py > 0e6179b8a7 > > ambari-server/src/main/resources/stacks/BigInsights/4.2/services/HDFS/metainfo.xml > 72ce2be363 > > ambari-server/src/main/resources/stacks/BigInsights/4.2/services/HDFS/package/scripts/hdfs.py > 2b158bd128 > > ambari-server/src/main/resources/stacks/BigInsights/4.2/services/HDFS/package/scripts/install_params.py > fe488
Review Request 63542: Add storage support for storing metric definitions using LevelDB
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63542/ --- Review request for Ambari and Aravindan Vijayan. Bugs: AMBARI-22365 https://issues.apache.org/jira/browse/AMBARI-22365 Repository: ambari Description --- Metrics Profile information storage requires reliable small volume embedded database that can store and retrieve metric definitions tracked by the Anomaly detection service. Diffs - ambari-metrics/ambari-metrics-anomaly-detection-service/pom.xml 44bdc1f ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/resources/config.yml 920c50c ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/app/AnomalyDetectionAppConfig.scala c1ef0d1 ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/app/AnomalyDetectionAppModule.scala 7425a7e ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/configuration/MetricDefinitionDBConfiguration.scala PRE-CREATION ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/db/MetadataDatasource.scala PRE-CREATION ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/leveldb/LevelDBDatasource.scala PRE-CREATION ambari-metrics/ambari-metrics-anomaly-detection-service/src/test/scala/org/apache/ambari/metrics/adservice/leveldb/LevelDBDataSourceTest.scala PRE-CREATION Diff: https://reviews.apache.org/r/63542/diff/1/ Testing --- Unit test added. Thanks, Sid Wagle
Re: Review Request 63524: Fix Serialization issues in Metric Definition Service
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63524/#review189967 --- Ship it! Ship It! - Sid Wagle On Nov. 2, 2017, 9:24 p.m., Aravindan Vijayan wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63524/ > --- > > (Updated Nov. 2, 2017, 9:24 p.m.) > > > Review request for Ambari, Jayush Luniya and Sid Wagle. > > > Repository: ambari > > > Description > --- > > Fix Serialization issues in Metric Definition Service > Rename Metric Manager Service to Metric Definition Service > Remove unneccesary setters in Config. > > > Diffs > - > > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/resources/config.yml > 6953745 > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/app/AnomalyDetectionAppConfig.scala > be8d027 > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/common/Season.scala > aba2587 > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/configuration/MetricCollectorConfiguration.scala > 50a0b72 > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/configuration/MetricDefinitionServiceConfiguration.scala > PRE-CREATION > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/configuration/MetricManagerServiceConfiguration.scala > e5960d5 > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/db/PhoenixAnomalyStoreAccessor.scala > 1191e90 > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/metadata/ADMetadataProvider.scala > 801c5f5 > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/metadata/MetricDefinition.scala > 0a5e51f > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/metadata/MetricDefinitionService.scala > PRE-CREATION > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/metadata/MetricDefinitionServiceImpl.scala > PRE-CREATION > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/metadata/MetricManagerService.scala > 12bd7e4 > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/metadata/MetricManagerServiceImpl.scala > ce02775 > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/metadata/MetricSourceDefinition.scala > 60198e0 > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/test/scala/org/apache/ambari/metrics/adservice/app/AnomalyDetectionAppConfigTest.scala > 8e3a949 > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/test/scala/org/apache/ambari/metrics/adservice/common/SeasonTest.scala > 4d542e8 > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/test/scala/org/apache/ambari/metrics/adservice/metadata/MetricDefinitionServiceTest.scala > PRE-CREATION > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/test/scala/org/apache/ambari/metrics/adservice/metadata/MetricManagerServiceTest.scala > 8e19a0f > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/test/scala/org/apache/ambari/metrics/adservice/metadata/MetricSourceDefinitionTest.scala > c4d639c > > > Diff: https://reviews.apache.org/r/63524/diff/1/ > > > Testing > --- > > Unit testing. > (Apache JIRA seems to be unreachable. Will create jira once it is up.) > > > Thanks, > > Aravindan Vijayan > >
Re: Review Request 63477: AMBARI-22348 : Metric Definition Service V1 Implementation.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63477/#review189837 --- Fix it, then Ship it! ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/resources/config.yml Lines 27 (patched) <https://reviews.apache.org/r/63477/#comment267065> /etc/ams-adservice, similar to others? ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/app/AnomalyDetectionAppConfig.scala Lines 25 (patched) <https://reviews.apache.org/r/63477/#comment267066> Make sure imports are properly alligned based on project rules. ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/app/AnomalyDetectionAppConfig.scala Lines 39 (patched) <https://reviews.apache.org/r/63477/#comment267067> Why var? ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/configuration/HBaseConfiguration.scala Lines 26 (patched) <https://reviews.apache.org/r/63477/#comment267068> Why var? ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/configuration/MetricManagerServiceConfiguration.scala Lines 36 (patched) <https://reviews.apache.org/r/63477/#comment267069> Is setter needed? ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/db/AdMetadataStoreAccessor.scala Lines 30 (patched) <https://reviews.apache.org/r/63477/#comment267070> This behavior should be left upto the impl and not a part of trait behavior ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/db/PhoenixAnomalyStoreAccessor.scala Lines 81 (patched) <https://reviews.apache.org/r/63477/#comment267071> Convoluted naming convention. ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/metricmanager/AMSMetadataProvider.scala Lines 32 (patched) <https://reviews.apache.org/r/63477/#comment267072> Rename to ADMetadataProvider ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/metricmanager/AMSMetadataProvider.scala Lines 34 (patched) <https://reviews.apache.org/r/63477/#comment267073> Prefer immutables here. ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/metricmanager/AMSMetadataProvider.scala Lines 54 (patched) <https://reviews.apache.org/r/63477/#comment267074> Unused? ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/metricmanager/MetricManagerServiceImpl.scala Lines 29 (patched) <https://reviews.apache.org/r/63477/#comment267077> Immutable should be preferred ! ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/metricmanager/MetricManagerServiceImpl.scala Lines 62 (patched) <https://reviews.apache.org/r/63477/#comment267078> Service initialization should not make a blocking http call rather fail late when API call is made. Log error when failing. ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricsService.java Lines 469 (patched) <https://reviews.apache.org/r/63477/#comment267079> formatting. - Sid Wagle On Nov. 1, 2017, 9:21 p.m., Aravindan Vijayan wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63477/ > --- > > (Updated Nov. 1, 2017, 9:21 p.m.) > > > Review request for Ambari, Jayush Luniya and Sid Wagle. > > > Bugs: AMBARI-22348 > https://issues.apache.org/jira/browse/AMBARI-22348 > > > Repository: ambari > > > Description > --- > > Initial implementation for > . Confuguration Handling for the AD Service > . Model classes for Anomalies, Metric Definition, Metric Key etc. > . Seasonality Handling > . Metric Definition Service > . Unit testing > > > Diffs > - > > ambari-metrics/ambari-metrics-anomaly-detection-service/pom.xml 554d026 > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/resources/config.yml > 9ca9e95 > > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/app/AnomalyDetectionAppConfig.scala > 9e6cc6d > > ambari-metrics/ambari-
Review Request 63449: Add ability in AMS to tee metrics to a set of configured Kafka brokers + Other changes
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63449/ --- Review request for Ambari and Aravindan Vijayan. Bugs: AMBARI-22343 https://issues.apache.org/jira/browse/AMBARI-22343 Repository: ambari Description --- *Goals*: * Add a config item in AMS to hold kafka-broker list. * Send metrics (List to kafka-broker on configured topic. Diffs - ambari-common/src/main/python/resource_management/libraries/functions/package_conditions.py 5a16061 ambari-metrics/ambari-metrics-anomaly-detection-service/pom.xml 554d026 ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/resources/config.yml 9ca9e95 ambari-metrics/ambari-metrics-timelineservice/pom.xml fd1c8bd ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java e218691 ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricConfiguration.java 026eaf5 ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/sink/ExternalSinkProvider.java 48887d9 ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/sink/HttpSinkProvider.java bb84c8a ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/sink/KafkaSinkProvider.java PRE-CREATION ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/source/DefaultInternalMetricsSourceProvider.java b97c39f ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/source/RawMetricsSource.java 967d819 ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-admanager-config.xml PRE-CREATION ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-admanager-env.xml PRE-CREATION ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml cd4ddfd ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/metainfo.xml d563bec ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/ams.py 2d81bf7 ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/ams_admanager.py PRE-CREATION ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py 8845e36 ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/status.py 0b24ac0 ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/status_params.py 6057a8b ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/templates/admanager_config.yaml.j2 PRE-CREATION Diff: https://reviews.apache.org/r/63449/diff/1/ Testing --- Thanks, Sid Wagle
Re: Review Request 63334: Some pre-upgrade checks do not work when upgrading from IOP to HDP-2.6.3
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63334/#review189332 --- Ship it! Ship It! - Sid Wagle On Oct. 26, 2017, 7:19 p.m., Nate Cole wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63334/ > --- > > (Updated Oct. 26, 2017, 7:19 p.m.) > > > Review request for Ambari, Jonathan Hurley and Sid Wagle. > > > Bugs: AMBARI-22320 > https://issues.apache.org/jira/browse/AMBARI-22320 > > > Repository: ambari > > > Description > --- > > We need to take into account a full stack upgrade and mark all the services > as upgrade=true. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/state/repository/VersionDefinitionXml.java > 7944de8e8c > > ambari-server/src/test/java/org/apache/ambari/server/state/repository/VersionDefinitionTest.java > 8433518691 > > > Diff: https://reviews.apache.org/r/63334/diff/1/ > > > Testing > --- > > Manual. Unit tests pending. > > > Thanks, > > Nate Cole > >
Re: Review Request 62989: AMBARI-22238 : Maven cleanup of ambari utility, logsearch and ambari metrics modules
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62989/#review188060 --- Ship it! Ship It! - Sid Wagle On Oct. 13, 2017, 8:50 p.m., Aravindan Vijayan wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/62989/ > --- > > (Updated Oct. 13, 2017, 8:50 p.m.) > > > Review request for Ambari, Jayush Luniya, Miklos Gergely, Oliver Szabo, and > Sid Wagle. > > > Bugs: AMBARI-22238 > https://issues.apache.org/jira/browse/AMBARI-22238 > > > Repository: ambari > > > Description > --- > > Remove > annotations-1.3.2.jar > tools-#.jar > jdk.tools-#.jar > checkstyle-#.jar > > > Diffs > - > > ambari-logsearch/ambari-logsearch-assembly/pom.xml 283179c > ambari-logsearch/ambari-logsearch-logfeeder/pom.xml 49122e8 > ambari-logsearch/ambari-logsearch-portal/pom.xml a439266 > ambari-metrics/ambari-metrics-assembly/pom.xml b6b9ad7 > ambari-metrics/ambari-metrics-common/pom.xml b52b1d9 > ambari-metrics/ambari-metrics-timelineservice/pom.xml 25373e7 > utility/pom.xml a708384 > > > Diff: https://reviews.apache.org/r/62989/diff/1/ > > > Testing > --- > > mvn clean test. > > > Thanks, > > Aravindan Vijayan > >
Re: Review Request 62941: [Preliminary] Setup an application server for hosting the AD System Manager
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62941/
---
(Updated Oct. 12, 2017, 7:10 p.m.)
Review request for Ambari and Aravindan Vijayan.
Bugs: AMBARI-22192
https://issues.apache.org/jira/browse/AMBARI-22192
Repository: ambari
Description
---
* Investigate Play Server / Dropwizard / Akka for HTTP server needs.
* Setup the server for configuration
* Simple startup ('do nothing' is good enough).
Diffs (updated)
-
ambari-metrics/ambari-metrics-anomaly-detection-service/pom.xml 6f8f8c1
ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/resources/config.yml
PRE-CREATION
ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/app/AnomalyDetectionApp.scala
PRE-CREATION
ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/app/AnomalyDetectionAppConfig.scala
PRE-CREATION
ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/app/AnomalyDetectionAppModule.scala
PRE-CREATION
ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/app/DefaultHealthCheck.scala
PRE-CREATION
ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/app/GuiceInjector.scala
PRE-CREATION
ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/common/PhoenixQueryConstants.scala
5e90d2b
ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/resource/AnomalyResource.scala
PRE-CREATION
ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/resource/RootResource.scala
PRE-CREATION
ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/service/ADQueryService.scala
PRE-CREATION
ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/service/ADQueryServiceImpl.scala
PRE-CREATION
ambari-metrics/ambari-metrics-anomaly-detection-service/src/test/scala/org/apache/ambari/metrics/adservice/app/DefaultADResourceSpecTest.scala
PRE-CREATION
ambari-metrics/ambari-metrics-anomaly-detection-service/src/test/scala/org/apache/ambari/metrics/adservice/app/DropwizardAppRuleHelper.scala
PRE-CREATION
ambari-metrics/ambari-metrics-anomaly-detection-service/src/test/scala/org/apache/ambari/metrics/adservice/app/DropwizardResourceTestRuleHelper.scala
PRE-CREATION
ambari-metrics/ambari-metrics-anomaly-detection-service/src/test/scala/org/apache/ambari/metrics/adservice/common/ADManagerConfigurationTest.scala
535dc9e
ambari-metrics/ambari-metrics-common/pom.xml e868557
ambari-metrics/pom.xml fb2997a
ambari-utility/pom.xml 1e86d40
Diff: https://reviews.apache.org/r/62941/diff/2/
Changes: https://reviews.apache.org/r/62941/diff/1-2/
Testing
---
Issue with drop wizard logging still to be resolved.
Thanks,
Sid Wagle
Review Request 62941: [Preliminary] Setup an application server for hosting the AD System Manager
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62941/
---
Review request for Ambari and Aravindan Vijayan.
Bugs: AMBARI-22192
https://issues.apache.org/jira/browse/AMBARI-22192
Repository: ambari
Description
---
* Investigate Play Server / Dropwizard / Akka for HTTP server needs.
* Setup the server for configuration
* Simple startup ('do nothing' is good enough).
Diffs
-
ambari-metrics/ambari-metrics-anomaly-detection-service/pom.xml 6f8f8c1
ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/resources/config.yml
PRE-CREATION
ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/app/AnomalyDetectionApp.scala
PRE-CREATION
ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/app/AnomalyDetectionAppConfig.scala
PRE-CREATION
ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/app/AnomalyDetectionAppModule.scala
PRE-CREATION
ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/app/DefaultHealthCheck.scala
PRE-CREATION
ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/app/GuiceInjector.scala
PRE-CREATION
ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/common/PhoenixQueryConstants.scala
5e90d2b
ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/resource/AnomalyResource.scala
PRE-CREATION
ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/resource/RootResource.scala
PRE-CREATION
ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/service/ADQueryService.scala
PRE-CREATION
ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/service/ADQueryServiceImpl.scala
PRE-CREATION
ambari-metrics/ambari-metrics-anomaly-detection-service/src/test/scala/org/apache/ambari/metrics/adservice/app/DefaultADResourceSpecTest.scala
PRE-CREATION
ambari-metrics/ambari-metrics-anomaly-detection-service/src/test/scala/org/apache/ambari/metrics/adservice/app/DropwizardAppRuleHelper.scala
PRE-CREATION
ambari-metrics/ambari-metrics-anomaly-detection-service/src/test/scala/org/apache/ambari/metrics/adservice/app/DropwizardResourceTestRuleHelper.scala
PRE-CREATION
ambari-metrics/ambari-metrics-anomaly-detection-service/src/test/scala/org/apache/ambari/metrics/adservice/common/ADManagerConfigurationTest.scala
535dc9e
ambari-metrics/ambari-metrics-common/pom.xml e868557
ambari-metrics/pom.xml fb2997a
ambari-utility/pom.xml 1e86d40
Diff: https://reviews.apache.org/r/62941/diff/1/
Testing
---
Issue with drop wizard logging still to be resolved.
Thanks,
Sid Wagle
Re: Review Request 62932: Deploy fails due to PU code related issues (post-merge)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62932/#review187819 --- Ship it! Ship It! - Sid Wagle On Oct. 12, 2017, 12:38 p.m., Andrew Onischuk wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/62932/ > --- > > (Updated Oct. 12, 2017, 12:38 p.m.) > > > Review request for Ambari, Attila Doroszlai, Myroslav Papirkovskyy, and Sid > Wagle. > > > Bugs: AMBARI-22217 > https://issues.apache.org/jira/browse/AMBARI-22217 > > > Repository: ambari > > > Description > --- > > > Diffs > - > > ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py > 2270dcef32 > > ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py > a6a05bf39b > > ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py > 26b0af0a99 > > ambari-common/src/main/python/resource_management/libraries/functions/stack_select.py > a76a6e327a > > ambari-common/src/main/python/resource_management/libraries/functions/stack_tools.py > d9233a3151 > > ambari-common/src/main/python/resource_management/libraries/script/script.py > 9a046b9404 > > ambari-server/src/main/resources/common-services/ATLAS/0.7.0.3.0/package/scripts/status_params.py > 852a9cb664 > > ambari-server/src/main/resources/common-services/KAFKA/0.10.0.3.0/package/scripts/params.py > 869c329a5e > > ambari-server/src/main/resources/common-services/KNOX/0.5.0.3.0/package/scripts/status_params.py > 3cbd920b23 > > ambari-server/src/main/resources/common-services/OOZIE/4.2.0.3.0/package/scripts/status_params.py > ce990cf8fa > > ambari-server/src/main/resources/common-services/RANGER/1.0.0.3.0/package/scripts/params.py > b88f1a4d96 > > ambari-server/src/main/resources/common-services/RANGER/1.0.0.3.0/package/scripts/status_params.py > 842430ba46 > > ambari-server/src/main/resources/common-services/RANGER_KMS/1.0.0.3.0/package/scripts/params.py > da8eb8ca40 > > ambari-server/src/main/resources/common-services/RANGER_KMS/1.0.0.3.0/package/scripts/status_params.py > 34d008208b > > ambari-server/src/main/resources/common-services/SQOOP/1.4.4.3.0/package/scripts/params_linux.py > 400c87c828 > > ambari-server/src/main/resources/common-services/STORM/1.0.1.3.0/package/scripts/status_params.py > d84b0950a9 > > ambari-server/src/main/resources/common-services/SUPERSET/0.15.0/package/scripts/params.py > b38d6438aa > ambari-server/src/test/python/custom_actions/test_ru_execute_tasks.py > 0d12a91276 > ambari-server/src/test/python/custom_actions/test_ru_set_all.py de83f7e07b > > > Diff: https://reviews.apache.org/r/62932/diff/1/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Andrew Onischuk > >
Re: Review Request 62805: AMBARI-22155. Intermittent failure of FlumeTimelineMetricsSinkTest
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62805/#review187278 --- Ship it! ambari-metrics/ambari-metrics-flume-sink/src/test/java/org/apache/hadoop/metrics2/sink/flume/FlumeTimelineMetricsSinkTest.java Lines 84 (patched) <https://reviews.apache.org/r/62805/#comment264197> Missing expectLastCall() ? - Sid Wagle On Oct. 6, 2017, 9:47 a.m., Attila Doroszlai wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/62805/ > --- > > (Updated Oct. 6, 2017, 9:47 a.m.) > > > Review request for Ambari, Aravindan Vijayan, Dmytro Sen, and Sid Wagle. > > > Bugs: AMBARI-22155 > https://issues.apache.org/jira/browse/AMBARI-22155 > > > Repository: ambari > > > Description > --- > > Mock the executor to eliminate nondeterminism caused by threads. > > > Diffs > - > > > ambari-metrics/ambari-metrics-flume-sink/src/test/java/org/apache/hadoop/metrics2/sink/flume/FlumeTimelineMetricsSinkTest.java > bd4ae6af88ee13f4ba6a3f14ac8c138b72d63c9d > > > Diff: https://reviews.apache.org/r/62805/diff/1/ > > > Testing > --- > > ``` > $ mvn -am -pl ambari-metrics/ambari-metrics-flume-sink clean test > ... > Tests run: 5, Failures: 0, Errors: 0, Skipped: 0 > ... > [INFO] Reactor Summary: > [INFO] > [INFO] Ambari Main SUCCESS [ 6.159 > s] > [INFO] Apache Ambari Project POM .. SUCCESS [ 0.036 > s] > [INFO] ambari-utility . SUCCESS [ 3.880 > s] > [INFO] ambari-metrics . SUCCESS [ 0.594 > s] > [INFO] Ambari Metrics Common .. SUCCESS [ 4.240 > s] > [INFO] Ambari Metrics Flume Sink .. SUCCESS [ 4.697 > s] > [INFO] > > [INFO] BUILD SUCCESS > ``` > > > Thanks, > > Attila Doroszlai > >
Re: Review Request 62624: AMBARI-22077: Create maven module and package structure for the anomaly detection engine
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62624/#review186475 --- Ship it! Ship It! - Sid Wagle On Sept. 27, 2017, 7 p.m., Aravindan Vijayan wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/62624/ > --- > > (Updated Sept. 27, 2017, 7 p.m.) > > > Review request for Ambari, Dmytro Sen, Jaimin Jetly, Jayush Luniya, and Sid > Wagle. > > > Bugs: AMBARI-22077 > https://issues.apache.org/jira/browse/AMBARI-22077 > > > Repository: ambari > > > Description > --- > > Create an empty maven module skeleton within ambari-metrics. This module will > contain the code for the following work items. > * Anomaly detection techniques > * Anomaly detection spark batch processing and spark streaming applications > > Work done > Combined the ambari-metrics-alertservice (Java) and ambari-metrics-spark > (scala) modules to a single module -> ambari-metrics-anomaly-detector. > Checked in latest versions of prototype. > > > Diffs > - > > ambari-metrics/ambari-metrics-alertservice/pom.xml 4db8a6a > > ambari-metrics/ambari-metrics-alertservice/src/main/java/org/apache/ambari/metrics/alertservice/prototype/AmbariServerInterface.java > b98f04c > > ambari-metrics/ambari-metrics-alertservice/src/main/java/org/apache/ambari/metrics/alertservice/prototype/MetricAnomalyDetectorTestInput.java > 490328a > > ambari-metrics/ambari-metrics-alertservice/src/main/java/org/apache/ambari/metrics/alertservice/prototype/MetricAnomalyTester.java > bff8120 > > ambari-metrics/ambari-metrics-alertservice/src/main/java/org/apache/ambari/metrics/alertservice/prototype/MetricKafkaProducer.java > 8023d15 > > ambari-metrics/ambari-metrics-alertservice/src/main/java/org/apache/ambari/metrics/alertservice/prototype/MetricSparkConsumer.java > 61b3dee > > ambari-metrics/ambari-metrics-alertservice/src/main/java/org/apache/ambari/metrics/alertservice/prototype/MetricsCollectorInterface.java > dab4a0a > > ambari-metrics/ambari-metrics-alertservice/src/main/java/org/apache/ambari/metrics/alertservice/prototype/PointInTimeADSystem.java > b3e7bd3 > > ambari-metrics/ambari-metrics-alertservice/src/main/java/org/apache/ambari/metrics/alertservice/prototype/RFunctionInvoker.java > 4fdf27d > > ambari-metrics/ambari-metrics-alertservice/src/main/java/org/apache/ambari/metrics/alertservice/prototype/TestSeriesInputRequest.java > 7485f01 > > ambari-metrics/ambari-metrics-alertservice/src/main/java/org/apache/ambari/metrics/alertservice/prototype/TrendADSystem.java > df36a4a > > ambari-metrics/ambari-metrics-alertservice/src/main/java/org/apache/ambari/metrics/alertservice/prototype/TrendMetric.java > 3bead8b > > ambari-metrics/ambari-metrics-alertservice/src/main/java/org/apache/ambari/metrics/alertservice/prototype/common/DataSeries.java > eb19857 > > ambari-metrics/ambari-metrics-alertservice/src/main/java/org/apache/ambari/metrics/alertservice/prototype/common/ResultSet.java > 101b0e9 > > ambari-metrics/ambari-metrics-alertservice/src/main/java/org/apache/ambari/metrics/alertservice/prototype/common/StatisticUtils.java > 4ea4ac5 > > ambari-metrics/ambari-metrics-alertservice/src/main/java/org/apache/ambari/metrics/alertservice/prototype/methods/AnomalyDetectionTechnique.java > 0b10b4b > > ambari-metrics/ambari-metrics-alertservice/src/main/java/org/apache/ambari/metrics/alertservice/prototype/methods/MetricAnomaly.java > da4f030 > > ambari-metrics/ambari-metrics-alertservice/src/main/java/org/apache/ambari/metrics/alertservice/prototype/methods/ema/EmaModel.java > a31410d > > ambari-metrics/ambari-metrics-alertservice/src/main/java/org/apache/ambari/metrics/alertservice/prototype/methods/ema/EmaModelLoader.java > 62749c1 > > ambari-metrics/ambari-metrics-alertservice/src/main/java/org/apache/ambari/metrics/alertservice/prototype/methods/ema/EmaTechnique.java > 52c6cf3 > > ambari-metrics/ambari-metrics-alertservice/src/main/java/org/apache/ambari/metrics/alertservice/prototype/methods/hsdev/HsdevTechnique.java > 04f4a73 > > ambari-metrics/ambari-metrics-alertservice/src/main/java/org/apache/ambari/metrics/alertservice/prototype/methods/kstest/KSTechnique.java > ff8dbcf > > ambari-metrics/ambari-metrics-alertservice/src/main/java/org/apache/ambari/metrics/alertservice/seriesgenerator/A
Re: Review Request 61886: AMBARI-21803. Implement STOMP endpoint for alert definitions
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61886/#review186309 --- Ship it! Ship It! - Sid Wagle On Sept. 25, 2017, 2:19 p.m., Attila Doroszlai wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/61886/ > --- > > (Updated Sept. 25, 2017, 2:19 p.m.) > > > Review request for Ambari, Andrew Onischuk, Myroslav Papirkovskyy, Sandor > Magyari, Sebastian Toader, and Sid Wagle. > > > Bugs: AMBARI-21803 > https://issues.apache.org/jira/browse/AMBARI-21803 > > > Repository: ambari > > > Description > --- > > * Added endpoint > * Created data structure to match expected JSON format > * Added @JsonProperty for all attributes (to explicitly specify names) > * Added @JsonInclude to avoid null values > * Extracted some common logic to parent classes > * Clarified `HostNotRegisteredException` > * Simplified some `equals()`/`hashCode()` methods > * Simplified map operations in `AgentSessionManager` > * Handle alert report with reduced content (no service/component names) > > > Diffs > - > > ambari-agent/src/main/python/ambari_agent/ClusterAlertDefinitionsCache.py > d3ab48e4da79cc7615bd5dedfec3754ffe5accc2 > ambari-agent/src/main/python/ambari_agent/Constants.py > 6a258d26a3a5d03ab76a70208321f13936a99dc7 > ambari-agent/src/main/python/ambari_agent/HeartbeatThread.py > b6e1aaf5c721bcd464458d5a83448b2f96c39d62 > > ambari-agent/src/main/python/ambari_agent/listeners/AlertDefinitionsEventListener.py > 91ae5de02f2788745ee87458c92c0845e6a258f4 > ambari-agent/src/test/python/ambari_agent/TestAgentStompResponses.py > 3c571ab89e5114c8eb1cd2dc7213ff53561c678e > > ambari-server/src/main/java/org/apache/ambari/server/HostNotRegisteredException.java > 9e7b3a7efbfe09bdfa8d1dc0203d9370da881cb1 > > ambari-server/src/main/java/org/apache/ambari/server/agent/AgentSessionManager.java > 1f64fc1d2800fa2b1a1c6d82a2ca855f9e30bbca > > ambari-server/src/main/java/org/apache/ambari/server/agent/AlertDefinitionCommand.java > 07029c76e0d6e4ee2df40f6c64746506f2309826 > > ambari-server/src/main/java/org/apache/ambari/server/agent/stomp/AgentClusterDataHolder.java > 0f0a4919ba7e1343c31c8397d5309542c2463287 > > ambari-server/src/main/java/org/apache/ambari/server/agent/stomp/AgentConfigsHolder.java > 0de686dfaad7a3c437fa5f9af5c1fa8abd4e61e8 > > ambari-server/src/main/java/org/apache/ambari/server/agent/stomp/AgentCurrentDataController.java > 7035f38cfbb7a75bd4cb5721a542ef832a805c99 > > ambari-server/src/main/java/org/apache/ambari/server/agent/stomp/AgentDataHolder.java > 635611bf6d325deb33cde671556647b9c264a716 > > ambari-server/src/main/java/org/apache/ambari/server/agent/stomp/AgentHostDataHolder.java > 792f10ef33363fd57579792f51901681ad83aa0b > > ambari-server/src/main/java/org/apache/ambari/server/agent/stomp/AlertDefinitionsHolder.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/agent/stomp/HostLevelParamsHolder.java > db093f85a7f304e895bad098a3ca5a62d7a3859d > > ambari-server/src/main/java/org/apache/ambari/server/agent/stomp/MetadataHolder.java > b7e680627f3fd42e87b7152f9ab9de29f63a44fa > > ambari-server/src/main/java/org/apache/ambari/server/agent/stomp/TopologyHolder.java > 0da5d831d546da98e2284b048b914f972cacfc05 > > ambari-server/src/main/java/org/apache/ambari/server/agent/stomp/dto/AlertCluster.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/agent/stomp/dto/MetadataCluster.java > 3ee5ad6d00b4c4374cba48e80d1cb410500ee586 > > ambari-server/src/main/java/org/apache/ambari/server/configuration/spring/ApiStompConfig.java > 170fa1704cd86c39db629aa74c54114d56b766dd > > ambari-server/src/main/java/org/apache/ambari/server/events/AgentConfigsUpdateEvent.java > 640be2d89bce55a9f58cbaf6b4e5296df9fc1a8c > > ambari-server/src/main/java/org/apache/ambari/server/events/AlertDefinitionUpdateHolder.java > e820401d460bc672596a4ede60abcd39646be49e > > ambari-server/src/main/java/org/apache/ambari/server/events/AlertDefinitionsUpdateEvent.java > cbe6e3703960ce0af9563f41b63d2a71427b4267 > > ambari-server/src/main/java/org/apache/ambari/server/events/AmbariUpdateEvent.java > 90aa704ba1530cd77a13b77db5183f398a4f659e > > ambari-server/src/main/java/org/apache/ambari/server/ev
Re: Review Request 62474: AMS: Add smart configs support for service
> On Sept. 21, 2017, 7:37 p.m., Aravindan Vijayan wrote: > > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-env.xml > > Lines 81 (patched) > > <https://reviews.apache.org/r/62474/diff/1/?file=1832385#file1832385line81> > > > > Should we allow minimum of 0? Or is that a normal practice in other > > services' enhanced configs? Fixed > On Sept. 21, 2017, 7:37 p.m., Aravindan Vijayan wrote: > > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml > > Line 789 (original), 803 (patched) > > <https://reviews.apache.org/r/62474/diff/1/?file=1832388#file1832388line811> > > > > Unrelated query, but shouldnt this be in ams-env? We would ideally need > > restart of metrics collector and monitors for this change. We need to open another Jira for this property to be handled correctly collector needs to tell the sink: Start writing to the monitor from now on. - Sid --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62474/#review185913 ------- On Sept. 21, 2017, 7:57 p.m., Sid Wagle wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/62474/ > --- > > (Updated Sept. 21, 2017, 7:57 p.m.) > > > Review request for Ambari, Aravindan Vijayan, Dmytro Sen, and Jaimin Jetly. > > > Bugs: AMBARI-22034 > https://issues.apache.org/jira/browse/AMBARI-22034 > > > Repository: ambari > > > Description > --- > > Mode, memory and some other settings are hard to hand edit and can use some > better UI tooling. > > > Diffs > - > > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricConfiguration.java > 44073ab > > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricsFilter.java > 1446ec2 > > ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricsFilterTest.java > 81da5c8 > > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-env.xml > 4d33661 > > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-hbase-env.xml > 9c4fc02 > > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-hbase-site.xml > 2c9378e > > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml > 1b085f6 > > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/metainfo.xml > 43c3685 > > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/themes/theme.json > PRE-CREATION > > > Diff: https://reviews.apache.org/r/62474/diff/2/ > > > Testing > --- > > Manually tested. > > > Thanks, > > Sid Wagle > >
Re: Review Request 62474: AMS: Add smart configs support for service
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62474/ --- (Updated Sept. 21, 2017, 7:57 p.m.) Review request for Ambari, Aravindan Vijayan, Dmytro Sen, and Jaimin Jetly. Changes --- Fixes based on comments. Bugs: AMBARI-22034 https://issues.apache.org/jira/browse/AMBARI-22034 Repository: ambari Description --- Mode, memory and some other settings are hard to hand edit and can use some better UI tooling. Diffs (updated) - ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricConfiguration.java 44073ab ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricsFilter.java 1446ec2 ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricsFilterTest.java 81da5c8 ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-env.xml 4d33661 ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-hbase-env.xml 9c4fc02 ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-hbase-site.xml 2c9378e ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml 1b085f6 ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/metainfo.xml 43c3685 ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/themes/theme.json PRE-CREATION Diff: https://reviews.apache.org/r/62474/diff/2/ Changes: https://reviews.apache.org/r/62474/diff/1-2/ Testing --- Manually tested. Thanks, Sid Wagle
Review Request 62474: AMS: Add smart configs support for service
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62474/ --- Review request for Ambari, Aravindan Vijayan, Dmytro Sen, and Jaimin Jetly. Bugs: AMBARI-22034 https://issues.apache.org/jira/browse/AMBARI-22034 Repository: ambari Description --- Mode, memory and some other settings are hard to hand edit and can use some better UI tooling. Diffs - ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricConfiguration.java 44073ab ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricsFilter.java 1446ec2 ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricsFilterTest.java 81da5c8 ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-env.xml 4d33661 ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-hbase-env.xml 9c4fc02 ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-hbase-site.xml 2c9378e ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml 1b085f6 ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/metainfo.xml 43c3685 ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/themes/theme.json PRE-CREATION Diff: https://reviews.apache.org/r/62474/diff/1/ Testing --- Manually tested. Thanks, Sid Wagle
Re: Review Request 62259: Upgrade to IOP 4.2.5 from IOP 4.1 failed with combined Solr host names longer than item_text column size in table upgrade_item
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62259/ --- (Updated Sept. 18, 2017, 5:36 p.m.) Review request for Ambari, Attila Doroszlai, Dmytro Grinenko, Jonathan Hurley, Jayush Luniya, Myroslav Papirkovskyy, and Vitalyi Brodetskyi. Changes --- Removed special handling for MySQL to fallback to LONGTEXT solumn type mapping. Bugs: AMBARI-21942 https://issues.apache.org/jira/browse/AMBARI-21942 Repository: ambari Description --- Basically the problem here happens when Ambari is attempting to insert into the following upgrade_itme:item_text table:column, INSERT INTO upgrade_item (upgrade_item_id, hosts, item_text, stage_id, state, tasks, upgrade_group_id) VALUES (?, ?, ?, ?, ?, ?, ?) The message string for SOLR includes all the SOLR node hostnames whose combined string length exceeds the column length defined for "item_text" of 1024 characters. Diffs - ambari-server/src/main/java/org/apache/ambari/server/orm/DBAccessorImpl.java 9564934 ambari-server/src/main/java/org/apache/ambari/server/orm/entities/UpgradeItemEntity.java 35ea769 ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog260.java 9e145c0 ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql b9c6985 ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql c53cd7b ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 6f754bb ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 19c0214 ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 8567d09 ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql fdbad09 Diff: https://reviews.apache.org/r/62259/diff/1/ Testing --- Manually verified the upgrade resulted in expansion on PostgreSQL. File Attachments (updated) Removed MySQL special handling https://reviews.apache.org/media/uploaded/files/2017/09/18/05ae5778-da1d-4ce6-971e-db5c639f2346__AMBARI-21942-2.patch Thanks, Sid Wagle
Re: Review Request 61510: AMBARI-21686 : Implement a test driver that provides a set of metric series with different kinds of metric behavior
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61510/#review185454 --- Ship it! Ship It! - Sid Wagle On Aug. 8, 2017, 10:14 p.m., Aravindan Vijayan wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/61510/ > --- > > (Updated Aug. 8, 2017, 10:14 p.m.) > > > Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle. > > > Bugs: AMBARI-21686 > https://issues.apache.org/jira/browse/AMBARI-21686 > > > Repository: ambari > > > Description > --- > > The generator is designed to generate the following series types. > > Uniform distributed series with some outliers. > Normally distributed series with some outliers. > Gradually increasing / decreasing series. (Monotonic Series) > Step Function Series > Series with a low value period and high value period (Not periodic) > Steady series with a small period of turbulence and then back to steady. > > > Diffs > - > > ambari-metrics/ambari-metrics-alertservice/pom.xml 4afc80f > > ambari-metrics/ambari-metrics-alertservice/src/main/java/org/apache/ambari/metrics/alertservice/R/AmsRTest.java > 2bbc250 > > ambari-metrics/ambari-metrics-alertservice/src/main/java/org/apache/ambari/metrics/alertservice/R/RFunctionInvoker.java > 2713b71 > > ambari-metrics/ambari-metrics-alertservice/src/main/java/org/apache/ambari/metrics/alertservice/common/DataSet.java > a709c73 > > ambari-metrics/ambari-metrics-alertservice/src/main/java/org/apache/ambari/metrics/alertservice/common/MethodResult.java > 6bf58df > > ambari-metrics/ambari-metrics-alertservice/src/main/java/org/apache/ambari/metrics/alertservice/common/MetricAnomaly.java > 4dbb425 > > ambari-metrics/ambari-metrics-alertservice/src/main/java/org/apache/ambari/metrics/alertservice/common/ResultSet.java > 9415c1b > > ambari-metrics/ambari-metrics-alertservice/src/main/java/org/apache/ambari/metrics/alertservice/common/SingleValuedTimelineMetric.java > acd4452 > > ambari-metrics/ambari-metrics-alertservice/src/main/java/org/apache/ambari/metrics/alertservice/common/StatisticUtils.java > 81bd77b > > ambari-metrics/ambari-metrics-alertservice/src/main/java/org/apache/ambari/metrics/alertservice/common/TimelineMetric.java > 88ad834 > > ambari-metrics/ambari-metrics-alertservice/src/main/java/org/apache/ambari/metrics/alertservice/common/TimelineMetrics.java > 7df6a9c > > ambari-metrics/ambari-metrics-alertservice/src/main/java/org/apache/ambari/metrics/alertservice/methods/MetricAnomalyModel.java > af33d26 > > ambari-metrics/ambari-metrics-alertservice/src/main/java/org/apache/ambari/metrics/alertservice/methods/ema/EmaDS.java > 32cd96b > > ambari-metrics/ambari-metrics-alertservice/src/main/java/org/apache/ambari/metrics/alertservice/methods/ema/EmaModel.java > 13a0f55 > > ambari-metrics/ambari-metrics-alertservice/src/main/java/org/apache/ambari/metrics/alertservice/methods/ema/EmaModelLoader.java > 0205844 > > ambari-metrics/ambari-metrics-alertservice/src/main/java/org/apache/ambari/metrics/alertservice/methods/ema/EmaResult.java > 2d24a9c > > ambari-metrics/ambari-metrics-alertservice/src/main/java/org/apache/ambari/metrics/alertservice/methods/ema/TestEmaModel.java > b851dab > > ambari-metrics/ambari-metrics-alertservice/src/main/java/org/apache/ambari/metrics/alertservice/prototype/AmbariServerInterface.java > PRE-CREATION > > ambari-metrics/ambari-metrics-alertservice/src/main/java/org/apache/ambari/metrics/alertservice/prototype/MetricAnomalyDetectorTestInput.java > PRE-CREATION > > ambari-metrics/ambari-metrics-alertservice/src/main/java/org/apache/ambari/metrics/alertservice/prototype/MetricAnomalyTester.java > PRE-CREATION > > ambari-metrics/ambari-metrics-alertservice/src/main/java/org/apache/ambari/metrics/alertservice/prototype/MetricKafkaProducer.java > PRE-CREATION > > ambari-metrics/ambari-metrics-alertservice/src/main/java/org/apache/ambari/metrics/alertservice/prototype/MetricSparkConsumer.java > PRE-CREATION > > ambari-metrics/ambari-metrics-alertservice/src/main/java/org/apache/ambari/metrics/alertservice/prototype/MetricsCollectorInterface.java > PRE-CREATION > > ambari-metrics/ambari-metrics-alertservice/src/main/java/org/apache/ambari/metrics/alertservice/prototype/PointInTimeADSystem.java > PRE-CREATION >
Re: Review Request 62294: Batch requests(Rolling Restart, HSI enable/disable, decommission/recommission) not working when Ambari is configured with Knox proxy
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62294/#review185358 --- Ship it! Ship It! - Sid Wagle On Sept. 13, 2017, 10:54 p.m., Jaimin Jetly wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/62294/ > --- > > (Updated Sept. 13, 2017, 10:54 p.m.) > > > Review request for Ambari, Myroslav Papirkovskyy, Sid Wagle, and Yusaku Sako. > > > Bugs: AMBARI-21944 > https://issues.apache.org/jira/browse/AMBARI-21944 > > > Repository: ambari > > > Description > --- > > Batch requests(Rolling Restart, HSI enable/disable, > decommission/recommission) not working when Ambari is configured with Knox > proxy > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/scheduler/ExecutionScheduleManager.java > f3b3c415ee > > ambari-server/src/test/java/org/apache/ambari/server/scheduler/ExecutionScheduleManagerTest.java > f901bda2b7 > ambari-web/app/controllers/main/host/details.js ad2ac98498 > ambari-web/app/controllers/main/service/item.js 224f83bdd9 > ambari-web/app/mixins/main/service/configs/component_actions_by_configs.js > 140e7de641 > ambari-web/app/utils/batch_scheduled_requests.js 6d1e721d47 > > ambari-web/test/mixins/main/service/configs/component_actions_by_configs_test.js > 9a73a9a3e1 > > > Diff: https://reviews.apache.org/r/62294/diff/2/ > > > Testing > --- > > Tested manually on a cluster. > verified that ambari-web unit test and ExecutionScheduleManagerTest test > passes successfully: > > 30490 passing (20s) > 157 pending > > > Thanks, > > Jaimin Jetly > >
Re: Review Request 62294: Batch requests(Rolling Restart, HSI enable/disable, decommission/recommission) not working when Ambari is configured with Knox proxy
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62294/#review185336 --- Fix it, then Ship it! ambari-server/src/main/java/org/apache/ambari/server/scheduler/ExecutionScheduleManager.java Line 665 (original), 669 (patched) <https://reviews.apache.org/r/62294/#comment261641> If ambariWebResource is a class level reference why make it a param? ambari-server/src/main/java/org/apache/ambari/server/scheduler/ExecutionScheduleManager.java Lines 803 (patched) <https://reviews.apache.org/r/62294/#comment261640> Why static? - Sid Wagle On Sept. 13, 2017, 8:25 p.m., Jaimin Jetly wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/62294/ > --- > > (Updated Sept. 13, 2017, 8:25 p.m.) > > > Review request for Ambari, Myroslav Papirkovskyy, Sid Wagle, and Yusaku Sako. > > > Bugs: AMBARI-21944 > https://issues.apache.org/jira/browse/AMBARI-21944 > > > Repository: ambari > > > Description > --- > > Batch requests(Rolling Restart, HSI enable/disable, > decommission/recommission) not working when Ambari is configured with Knox > proxy > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/scheduler/ExecutionScheduleManager.java > f3b3c415ee > > ambari-server/src/test/java/org/apache/ambari/server/scheduler/ExecutionScheduleManagerTest.java > f901bda2b7 > ambari-web/app/controllers/main/host/details.js ad2ac98498 > ambari-web/app/controllers/main/service/item.js 224f83bdd9 > ambari-web/app/mixins/main/service/configs/component_actions_by_configs.js > 140e7de641 > ambari-web/app/utils/batch_scheduled_requests.js 6d1e721d47 > > ambari-web/test/mixins/main/service/configs/component_actions_by_configs_test.js > 9a73a9a3e1 > > > Diff: https://reviews.apache.org/r/62294/diff/1/ > > > Testing > --- > > Tested manually on a cluster. > verified that ambari-web unit test and ExecutionScheduleManagerTest test > passes successfully: > > 30490 passing (20s) > 157 pending > > > Thanks, > > Jaimin Jetly > >
Re: Review Request 62292: AMBARI-21952 : Refine AMS heap recommendations in stack advisor.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62292/#review185333 --- Ship it! ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py Lines 873 (patched) <https://reviews.apache.org/r/62292/#comment261637> Is this left behind artifact? - Sid Wagle On Sept. 13, 2017, 7:49 p.m., Aravindan Vijayan wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/62292/ > --- > > (Updated Sept. 13, 2017, 7:49 p.m.) > > > Review request for Ambari, Dmytro Sen, Jayush Luniya, and Sid Wagle. > > > Bugs: AMBARI-21952 > https://issues.apache.org/jira/browse/AMBARI-21952 > > > Repository: ambari > > > Description > --- > > Refined stack advisor recommendations for AMS heap sizing. > Added validation for 'distributed' mode when number of sinks > 1000. > Added validation for HBase heapsizes > 32G > Removed unneeded validations for gouging unused memory on metrics collector > host. > > > Diffs > - > > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py > 7992c95 > ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py > 35af503 > ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py > 9c6bef1 > > > Diff: https://reviews.apache.org/r/62292/diff/1/ > > > Testing > --- > > Manually tested. > Python unit tests passed. > > > File Attachments > > > Heap estimates for various size clusters > > https://reviews.apache.org/media/uploaded/files/2017/09/13/ce769beb-f026-4d9e-9530-3218327584be__Screen_Shot_2017-09-13_at_12.13.24_PM.png > > > Thanks, > > Aravindan Vijayan > >
Re: Review Request 62250: AMBARI-21939 : Ambari HDFS Heatmaps are not showing data (Not Applicable) for bytes read, bytes written & DataNode Process Network I/O Utilization
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62250/#review185246 --- Ship it! Ship It! - Sid Wagle On Sept. 12, 2017, 4:58 p.m., Aravindan Vijayan wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/62250/ > --- > > (Updated Sept. 12, 2017, 4:58 p.m.) > > > Review request for Ambari, Dmytro Sen, Jayush Luniya, and Sid Wagle. > > > Bugs: AMBARI-21939 > https://issues.apache.org/jira/browse/AMBARI-21939 > > > Repository: ambari > > > Description > --- > > PROBLEM: HDFS Heatmaps are not showing data (Not Applicable) for bytes read, > bytes written & DataNode Process Network I/O Utilization > > EXPECTED BEHAVIOUR: Heatmaps should show data > > ACTUAL BEHAVIOUR: Data is being shown as Not Applicable (Grey Box) > > ANALYSIS > This is reproducible in Ambari 2.2.2 and Ambari 2.5.1.0. If you go to HDFS, > then click on Heatmaps, you can see the API call go to > fields=host_components/metrics/dfs/datanode/bytes_written._rate > however this returns no values. If you remove ._rate then you can see the > values which you would expect. > Attach one or more files to this issue > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog260.java > 9e145c0 > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/widgets.json > bcfb2cc > ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/widgets.json > eeb9ff8 > > ambari-server/src/main/resources/stacks/PERF/1.0/services/FAKEHDFS/widgets.json > 7a793f8 > > ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog260Test.java > db6ebc1 > > > Diff: https://reviews.apache.org/r/62250/diff/1/ > > > Testing > --- > > Manually tested. > UpgradeCatalog260 test added. > > > Thanks, > > Aravindan Vijayan > >
Re: Review Request 62125: AMBARI-21893 : NameNode Heap Usage (Daily) metric alert status flips to UNKNOWN intermittently when AMS HTTPS is enabled.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62125/#review185245 --- Fix it, then Ship it! ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py Lines 101 (patched) <https://reviews.apache.org/r/62125/#comment261548> More redable as host.alias vs the other way. - Sid Wagle On Sept. 12, 2017, 7:21 p.m., Aravindan Vijayan wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/62125/ > --- > > (Updated Sept. 12, 2017, 7:21 p.m.) > > > Review request for Ambari, Dmytro Sen, Jayush Luniya, and Sid Wagle. > > > Bugs: AMBARI-21893 > https://issues.apache.org/jira/browse/AMBARI-21893 > > > Repository: ambari > > > Description > --- > > PROBLEM > This issue is because of AMS HTTPS + HA scenario. The CA cert file > (/etc/ambari-metrics-monitor/conf/ca.pem) found on every host is generated by > fetching a specific metric collector host's certificate from the truststore. > This certificate file is being used by alert script, service check and even > monitors to talk to collector. > > For example, in a cluster with hosts H1 to H5, let's say there are 2 > collectors - H1 & H2. > On a node H3, let's say the ca.pem file was constructed using the certificate > for collector H2. > Service check or metric monitor on H3 will NOT be able to talk to H1 since it > does not have the certificate for that host. > > FIX > Import certificates from metric collectors onto the pem file, if > metric_truststore_alias is not specified. If the cluster has multiple > collectors and the same certificate is used for all the collectors, then the > metric_truststore_alias must be used to specify the alias to import the > certificate. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog260.java > 9e145c0 > > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-ssl-client.xml > cac39de > > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/ams.py > 4672501 > > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py > 071882b > > ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog260Test.java > db6ebc1 > > > Diff: https://reviews.apache.org/r/62125/diff/2/ > > > Testing > --- > > Manually tested. > python unit tests pass. > > > Thanks, > > Aravindan Vijayan > >
Re: Review Request 62186: Implement host recovery - backend changes
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62186/#review184988 --- Ship it! Ship It! - Sid Wagle On Sept. 8, 2017, 4:17 p.m., Eugene Chekanskiy wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/62186/ > --- > > (Updated Sept. 8, 2017, 4:17 p.m.) > > > Review request for Ambari, Dmytro Sen, Robert Levas, and Sid Wagle. > > > Bugs: AMBARI-21868 > https://issues.apache.org/jira/browse/AMBARI-21868 > > > Repository: ambari > > > Description > --- > > see jira > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/api/resources/ClusterResourceDefinition.java > 8933dd3 > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java > 96280ea > > ambari-server/src/main/java/org/apache/ambari/server/controller/DeleteIdentityHandler.java > a7b9d80 > > ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelper.java > bb360b5 > > ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelperImpl.java > 013a063 > > ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/AbstractPrepareKerberosServerAction.java > d6b8ffc > > ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/CreateKeytabFilesServerAction.java > a23ab5d > > ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/CreatePrincipalsServerAction.java > 2fd5abe > > ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/KerberosIdentityDataFile.java > 81e345a > > ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/KerberosIdentityDataFileWriter.java > f55c6f4 > > ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/PrepareDisableKerberosServerAction.java > 4e63f4a > > ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/PrepareEnableKerberosServerAction.java > e13f033 > > ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/PrepareKerberosIdentitiesServerAction.java > 00c82a5 > > ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/PreconfigureKerberosAction.java > 697f1d1 > > ambari-server/src/test/java/org/apache/ambari/server/agent/TestHeartbeatHandler.java > b4ff5c1 > > ambari-server/src/test/java/org/apache/ambari/server/serveraction/kerberos/AbstractPrepareKerberosServerActionTest.java > 95e5513 > > ambari-server/src/test/java/org/apache/ambari/server/serveraction/kerberos/KerberosIdentityDataFileTest.java > 323ba8e > > ambari-server/src/test/java/org/apache/ambari/server/serveraction/kerberos/KerberosServerActionTest.java > f63e6b8 > > > Diff: https://reviews.apache.org/r/62186/diff/2/ > > > Testing > --- > > mvn clean test; cluster deploy > > > Thanks, > > Eugene Chekanskiy > >
Review Request 62169: Server returns 500 error for create config group request
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62169/
---
Review request for Ambari, Aravindan Vijayan, Jonathan Hurley, and Swapan
Shridhar.
Bugs: AMBARI-21908
https://issues.apache.org/jira/browse/AMBARI-21908
Repository: ambari
Description
---
Cannot create config group, server returns 500 error.
Form data:
{code}
{noformat}
[{"ConfigGroup":{"group_name":"11","tag":"HDFS","description":"","desired_configs":[],"hosts":[]},"serviceName":"HDFS"}]
{noformat}
{code}
Exception:
{noformat}
{
"status" : 400,
"message" : "The properties [ConfigGroup/serviceName] specified in the
request or predicate are not supported for the resource type ConfigGroup."
}
{noformat}
Diffs
-
ambari-server/src/main/java/org/apache/ambari/server/controller/ConfigGroupRequest.java
efa1a7e
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ConfigGroupResourceProvider.java
0ea54cb
ambari-server/src/main/java/org/apache/ambari/server/topology/AmbariContext.java
da517f3
ambari-server/src/main/resources/properties.json 80a86b8
Diff: https://reviews.apache.org/r/62169/diff/1/
Testing
---
Manually verified.
Affected unit tests are green.
Thanks,
Sid Wagle
Re: Review Request 62134: Property provider in-memory maps are refreshed too slowly after config updates
> On Sept. 7, 2017, 6:23 p.m., Jonathan Hurley wrote: > > We need an @WTF annotation for stuff like this ... :D +1 - Sid --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62134/#review184854 --- On Sept. 7, 2017, 6:19 p.m., Sid Wagle wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/62134/ > --- > > (Updated Sept. 7, 2017, 6:19 p.m.) > > > Review request for Ambari, Jonathan Hurley, Nate Cole, Robert Levas, and > Sebastian Toader. > > > Bugs: AMBARI-21898 > https://issues.apache.org/jira/browse/AMBARI-21898 > > > Repository: ambari > > > Description > --- > > Building the hostComponentMap map for ProviderModule seems does unnecessary > computation going through the ResourceProvider layer. On every configuration > update, we call the reset on these maps which takes a long time to refresh. > Instead, relying on > org.apache.ambari.server.state.cluster.ClusterImpl#serviceComponentHosts will > lead to faster computation without overhead like stale config calculations > for every SCH. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractProviderModule.java > 5fc4c31 > ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java > ee18fdf > > ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java > 2f858b8 > > > Diff: https://reviews.apache.org/r/62134/diff/3/ > > > Testing > --- > > PropertyProvider tests are all green > Pending unit tests and manual testing. > > > Thanks, > > Sid Wagle > >
Re: Review Request 62134: Property provider in-memory maps are refreshed too slowly after config updates
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62134/ --- (Updated Sept. 7, 2017, 6:19 p.m.) Review request for Ambari, Jonathan Hurley, Nate Cole, Robert Levas, and Sebastian Toader. Changes --- Fixed the NPE caused due to the fact that DefaultProviderModule is not instantiated trough guice and thereby the field injections do not really work. The contructor actually initializes these fields explcitily. Verified the fix manually and also added a TODO for future. Bugs: AMBARI-21898 https://issues.apache.org/jira/browse/AMBARI-21898 Repository: ambari Description --- Building the hostComponentMap map for ProviderModule seems does unnecessary computation going through the ResourceProvider layer. On every configuration update, we call the reset on these maps which takes a long time to refresh. Instead, relying on org.apache.ambari.server.state.cluster.ClusterImpl#serviceComponentHosts will lead to faster computation without overhead like stale config calculations for every SCH. Diffs (updated) - ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractProviderModule.java 5fc4c31 ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java ee18fdf ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java 2f858b8 Diff: https://reviews.apache.org/r/62134/diff/3/ Changes: https://reviews.apache.org/r/62134/diff/2-3/ Testing --- PropertyProvider tests are all green Pending unit tests and manual testing. Thanks, Sid Wagle
Re: Review Request 62134: Property provider in-memory maps are refreshed too slowly after config updates
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62134/ --- (Updated Sept. 6, 2017, 10:08 p.m.) Review request for Ambari, Jonathan Hurley, Robert Levas, and Sebastian Toader. Changes --- Fix for initialization of the portMap. Summary (updated) - Property provider in-memory maps are refreshed too slowly after config updates Bugs: AMBARI-21898 https://issues.apache.org/jira/browse/AMBARI-21898 Repository: ambari Description --- Building the hostComponentMap map for ProviderModule seems does unnecessary computation going through the ResourceProvider layer. On every configuration update, we call the reset on these maps which takes a long time to refresh. Instead, relying on org.apache.ambari.server.state.cluster.ClusterImpl#serviceComponentHosts will lead to faster computation without overhead like stale config calculations for every SCH. Diffs (updated) - ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractProviderModule.java 5fc4c31 ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java ee18fdf ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java 2f858b8 Diff: https://reviews.apache.org/r/62134/diff/2/ Changes: https://reviews.apache.org/r/62134/diff/1-2/ Testing --- PropertyProvider tests are all green Pending unit tests and manual testing. Thanks, Sid Wagle
Review Request 62134: [DRAFT] Property provider in-memory maps are refreshed too slowly after config updates
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62134/ --- Review request for Ambari, Jonathan Hurley, Robert Levas, and Sebastian Toader. Bugs: AMBARI-21898 https://issues.apache.org/jira/browse/AMBARI-21898 Repository: ambari Description --- Building the hostComponentMap map for ProviderModule seems does unnecessary computation going through the ResourceProvider layer. On every configuration update, we call the reset on these maps which takes a long time to refresh. Instead, relying on org.apache.ambari.server.state.cluster.ClusterImpl#serviceComponentHosts will lead to faster computation without overhead like stale config calculations for every SCH. Diffs - ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractProviderModule.java 5fc4c31 ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java ee18fdf ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java 2f858b8 Diff: https://reviews.apache.org/r/62134/diff/1/ Testing --- PropertyProvider tests are all green Pending unit tests and manual testing. Thanks, Sid Wagle
Re: Review Request 62125: AMBARI-21893 : NameNode Heap Usage (Daily) metric alert status flips to UNKNOWN intermittently when AMS HTTPS is enabled.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62125/#review184719 --- ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/ams.py Lines 519 (patched) <https://reviews.apache.org/r/62125/#comment260911> This assumes that alais is hostname we should allow a config of type prefix.cert.hostname.alias to allow alias mapped to cert coming from the collector. - Sid Wagle On Sept. 6, 2017, 5:31 p.m., Aravindan Vijayan wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/62125/ > --- > > (Updated Sept. 6, 2017, 5:31 p.m.) > > > Review request for Ambari, Dmytro Sen, Jayush Luniya, and Sid Wagle. > > > Bugs: AMBARI-21893 > https://issues.apache.org/jira/browse/AMBARI-21893 > > > Repository: ambari > > > Description > --- > > PROBLEM > This issue is because of AMS HTTPS + HA scenario. The CA cert file > (/etc/ambari-metrics-monitor/conf/ca.pem) found on every host is generated by > fetching a specific metric collector host's certificate from the truststore. > This certificate file is being used by alert script, service check and even > monitors to talk to collector. > > For example, in a cluster with hosts H1 to H5, let's say there are 2 > collectors - H1 & H2. > On a node H3, let's say the ca.pem file was constructed using the certificate > for collector H2. > Service check or metric monitor on H3 will NOT be able to talk to H1 since it > does not have the certificate for that host. > > FIX > Import certificates from metric collectors onto the pem file, if > metric_truststore_alias is not specified. If the cluster has multiple > collectors and the same certificate is used for all the collectors, then the > metric_truststore_alias must be used to specify the alias to import the > certificate. > > > Diffs > - > > > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/ams.py > 9a31ade > > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py > 6975bec > > > Diff: https://reviews.apache.org/r/62125/diff/1/ > > > Testing > --- > > Manually tested. > python unit tests pass. > > > Thanks, > > Aravindan Vijayan > >
Re: Review Request 62097: AMBARI-21871 : Add new graphs to HBase RegionServer dashboard in Grafana
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62097/#review184583 --- Ship it! Ship It! - Sid Wagle On Sept. 5, 2017, 9:16 p.m., Aravindan Vijayan wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/62097/ > --- > > (Updated Sept. 5, 2017, 9:16 p.m.) > > > Review request for Ambari, Dmytro Sen, Jayush Luniya, Sid Wagle, and Vivek > Ratnavel Subramanian. > > > Bugs: AMBARI-21871 > https://issues.apache.org/jira/browse/AMBARI-21871 > > > Repository: ambari > > > Description > --- > > HBASE-15160 and HBASE-18184 added some new HDFS-level metrics to HBase > RegionServers which would be good to expose in the HBase RegionServers > dashboard in grafana. > > Updated metric whitelist with newer metric list from Grafana dashboards. > > > Diffs > - > > ambari-metrics/ambari-metrics-timelineservice/conf/unix/metrics_whitelist > bd36429 > > ambari-metrics/ambari-metrics-timelineservice/conf/windows/metrics_whitelist > bd36429 > > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/files/grafana-dashboards/HDP/grafana-hbase-regionservers.json > 3184357 > > > Diff: https://reviews.apache.org/r/62097/diff/1/ > > > Testing > --- > > Manually tested. > > > Thanks, > > Aravindan Vijayan > >
Re: Review Request 62072: Remove hive.custom-extensions.root from Hive Properties.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62072/#review184574 --- Ship it! Ship It! - Sid Wagle On Sept. 5, 2017, 6:46 p.m., Vitalyi Brodetskyi wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/62072/ > --- > > (Updated Sept. 5, 2017, 6:46 p.m.) > > > Review request for Ambari, Andrew Onischuk and Sid Wagle. > > > Bugs: AMBARI-21876 > https://issues.apache.org/jira/browse/AMBARI-21876 > > > Repository: ambari > > > Description > --- > > Remove hive.custom-extensions.root from hive properties. > > > Diffs > - > > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/config-upgrade.xml > 4b53c7c > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/nonrolling-upgrade-to-hdp-2.6.xml > 48d1d92 > > ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/config-upgrade.xml > 29fcac0 > > ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/nonrolling-upgrade-to-hdp-2.6.xml > 1d4a6f4 > > ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/custom_extensions.py > 63f6f70 > > ambari-server/src/main/resources/stacks/HDP/2.6/services/HIVE/configuration/hive-site.xml > c0800ed > > > Diff: https://reviews.apache.org/r/62072/diff/4/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Vitalyi Brodetskyi > >
Re: Review Request 62072: Remove hive.custom-extensions.root from Hive Properties.
> On Sept. 5, 2017, 5:18 p.m., Sid Wagle wrote: > > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog260.java > > Lines 379 (patched) > > <https://reviews.apache.org/r/62072/diff/3/?file=1814453#file1814453line379> > > > > Why is this done during Ambari upgrade as well as stack upgrade? The > > presense of this property on existing stack should not cause ay issues > > after Amabri upgrade. > > Vitalyi Brodetskyi wrote: > Yes. it will not cause any issues. But as i understood, according to > first comment in jira "No issues, it’s just a warning. If Ambari sets this > config for Hive it should be fixed to not set it; this config does not > exist." they don't want hive to throw even WARN messages. And i'm not sure > about changes for BigInsight (stack upgrade), probably we don't need them, > what do you think? The remove from upgrade pack seems sufficient to me. As long as default 4.2 and 4.2.5 installs of IOP does not contain this property we can safely skip the UpgradeCatalog changes. - Sid --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62072/#review184559 --- On Sept. 5, 2017, 3:06 p.m., Vitalyi Brodetskyi wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/62072/ > ------- > > (Updated Sept. 5, 2017, 3:06 p.m.) > > > Review request for Ambari, Andrew Onischuk and Sid Wagle. > > > Bugs: AMBARI-21876 > https://issues.apache.org/jira/browse/AMBARI-21876 > > > Repository: ambari > > > Description > --- > > Remove hive.custom-extensions.root from hive properties. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog260.java > 5d50f29 > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/config-upgrade.xml > 4b53c7c > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/nonrolling-upgrade-to-hdp-2.6.xml > 48d1d92 > > ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/config-upgrade.xml > 29fcac0 > > ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/nonrolling-upgrade-to-hdp-2.6.xml > 1d4a6f4 > > ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/custom_extensions.py > 63f6f70 > > ambari-server/src/main/resources/stacks/HDP/2.6/services/HIVE/configuration/hive-site.xml > c0800ed > > ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog260Test.java > d8e8171 > > > Diff: https://reviews.apache.org/r/62072/diff/3/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Vitalyi Brodetskyi > >
Re: Review Request 62072: Remove hive.custom-extensions.root from Hive Properties.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62072/#review184559 --- ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog260.java Lines 379 (patched) <https://reviews.apache.org/r/62072/#comment260735> Why is this done during Ambari upgrade as well as stack upgrade? The presense of this property on existing stack should not cause ay issues after Amabri upgrade. - Sid Wagle On Sept. 5, 2017, 3:06 p.m., Vitalyi Brodetskyi wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/62072/ > --- > > (Updated Sept. 5, 2017, 3:06 p.m.) > > > Review request for Ambari, Andrew Onischuk and Sid Wagle. > > > Bugs: AMBARI-21876 > https://issues.apache.org/jira/browse/AMBARI-21876 > > > Repository: ambari > > > Description > --- > > Remove hive.custom-extensions.root from hive properties. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog260.java > 5d50f29 > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/config-upgrade.xml > 4b53c7c > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/nonrolling-upgrade-to-hdp-2.6.xml > 48d1d92 > > ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/config-upgrade.xml > 29fcac0 > > ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/nonrolling-upgrade-to-hdp-2.6.xml > 1d4a6f4 > > ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/custom_extensions.py > 63f6f70 > > ambari-server/src/main/resources/stacks/HDP/2.6/services/HIVE/configuration/hive-site.xml > c0800ed > > ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog260Test.java > d8e8171 > > > Diff: https://reviews.apache.org/r/62072/diff/3/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Vitalyi Brodetskyi > >
Re: Review Request 61969: Database consistency check fails after upgrade to ambari 2.6.0
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61969/#review184066 --- Ship it! Ship It! - Sid Wagle On Aug. 29, 2017, 2:55 p.m., Vitalyi Brodetskyi wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/61969/ > --- > > (Updated Aug. 29, 2017, 2:55 p.m.) > > > Review request for Ambari, Jonathan Hurley and Sid Wagle. > > > Bugs: AMBARI-21843 > https://issues.apache.org/jira/browse/AMBARI-21843 > > > Repository: ambari > > > Description > --- > > Here we need to fix two issues: > 1) Table "clusterconfigmapping" was removed in ambari-2.6.0 (dropping it > during upgrade). But db consistency check still using this table to get some > data. After upgrade it fails because "clusterconfigmapping" doesn't exist. > Need to reimplement checks which are using this table. > 2) Db consistency check doesn't show any information for user about errors, > in casse when table doesn't exist. You can check that with > "clusterconfigmapping" > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java > aa96875 > > ambari-server/src/test/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelperTest.java > 5ce03a3 > > > Diff: https://reviews.apache.org/r/61969/diff/2/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Vitalyi Brodetskyi > >
Re: Review Request 61945: Sqoop: hive import fails with exception 'java.io.IOException:java.lang.RuntimeException: native-lzo library not available' for LZO compression codec
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61945/#review183934 --- Ship it! Ship It! - Sid Wagle On Aug. 28, 2017, 3:17 p.m., Andrew Onischuk wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/61945/ > --- > > (Updated Aug. 28, 2017, 3:17 p.m.) > > > Review request for Ambari, Jonathan Hurley, Mahadev Konar, and Sid Wagle. > > > Bugs: AMBARI-21829 > https://issues.apache.org/jira/browse/AMBARI-21829 > > > Repository: ambari > > > Description > --- > > Sqoop: hive import fails with exception > 'java.io.IOException:java.lang.RuntimeException: native-lzo library not > available' for LZO compression codec > > > > > [hrt_qa@nat-r6-ddfu-sqoop-5 hadoopqe]$ > /usr/hdp/current/hive-client/bin/hive -e "select data from mytable_compress > where id=12;" > 2017-08-25 13:44:40,090 WARN [main] conf.HiveConf: HiveConf of name > hive.optimize.mapjoin.mapreduce does not exist > 2017-08-25 13:44:40,091 WARN [main] conf.HiveConf: HiveConf of name > hive.custom-extensions.root does not exist > > Logging initialized using configuration in > file:/etc/hive/2.6.2.0-200/0/hive-log4j.properties > OK > Failed with exception java.io.IOException:java.lang.RuntimeException: > native-lzo library not available > Time taken: 6.933 seconds > > > Diffs > - > > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/config-upgrade.xml > f90a20e > > ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/config-upgrade.xml > d85c7ee > > > Diff: https://reviews.apache.org/r/61945/diff/2/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Andrew Onischuk > >
Re: Review Request 61856: Auto fix service config mapping issue discovered during ambari-upgrade
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61856/#review183864 --- Fix it, then Ship it! ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog260.java Lines 166 (patched) <https://reviews.apache.org/r/61856/#comment259928> Does this comment still apply? - Sid Wagle On Aug. 25, 2017, 3:34 p.m., Vitalyi Brodetskyi wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/61856/ > --- > > (Updated Aug. 25, 2017, 3:34 p.m.) > > > Review request for Ambari, Myroslav Papirkovskyy and Sid Wagle. > > > Bugs: AMBARI-21795 > https://issues.apache.org/jira/browse/AMBARI-21795 > > > Repository: ambari > > > Description > --- > > If user has deleted services, after upgrade db consistency check will throw > warnings about unmapped configs. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java > 9592209 > > ambari-server/src/main/java/org/apache/ambari/server/orm/entities/ClusterConfigEntity.java > 34f3034 > ambari-server/src/main/java/org/apache/ambari/server/state/ServiceImpl.java > 319bf53 > > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog260.java > cee05ef > ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql 7348d4f > ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 28ac59e > ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql f31b308 > ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 8ed9da5 > ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 2a45476 > ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql c5d2e90 > > ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog260Test.java > 2e38686 > > > Diff: https://reviews.apache.org/r/61856/diff/4/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Vitalyi Brodetskyi > >
Re: Review Request 61833: Ambari server logs consistency check warning after service delete with ConfigGroup present
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61833/
---
(Updated Aug. 24, 2017, 6:46 p.m.)
Review request for Ambari, Jonathan Hurley, Myroslav Papirkovskyy, Sumit
Mohanty, and Sebastian Toader.
Changes
---
Addressed concerns from Jonathan regarding unnecessary logging in the main code
path and moved the check aswell as fix to db checker.
Bugs: AMBARI-21784
https://issues.apache.org/jira/browse/AMBARI-21784
Repository: ambari
Description
---
Ambari server log is constantly populated with 'Config inconsistency exists:
unknown configType=solr-site' even after Solr deletion due to existence of
config group.
{code}
22 Aug 2017 19:00:20,094 ERROR [ambari-hearbeat-monitor] HostImpl:1085 - Config
inconsistency exists: unknown configType=solr-site
{code}
Diffs (updated)
-
ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java
40049c0
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ConfigGroupResourceProvider.java
200cf27
ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
3fb45ab
ambari-server/src/main/java/org/apache/ambari/server/state/configgroup/ConfigGroupFactory.java
906d948
ambari-server/src/main/java/org/apache/ambari/server/state/configgroup/ConfigGroupImpl.java
9058a63
ambari-server/src/main/java/org/apache/ambari/server/state/host/HostImpl.java
7282485
ambari-server/src/test/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelperTest.java
9e85fa2
ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerTest.java
5214f04
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ConfigGroupResourceProviderTest.java
eec379e
ambari-server/src/test/java/org/apache/ambari/server/state/ConfigGroupTest.java
c70dcc6
ambari-server/src/test/java/org/apache/ambari/server/state/ConfigHelperTest.java
21ce2c2
ambari-server/src/test/java/org/apache/ambari/server/state/ServiceTest.java
1ad8cef
ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClusterTest.java
e8e7206
ambari-server/src/test/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostTest.java
fc1f334
Diff: https://reviews.apache.org/r/61833/diff/4/
Changes: https://reviews.apache.org/r/61833/diff/3-4/
Testing
---
Manually verfied with a 2.5.2 cluster.
Unit test added.
All modified unit test pass.
Thanks,
Sid Wagle
Re: Review Request 61833: Ambari server logs consistency check warning after service delete with ConfigGroup present
> On Aug. 24, 2017, 1:30 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
> > Lines 1428-1431 (original), 1444-1452 (patched)
> > <https://reviews.apache.org/r/61833/diff/3/?file=1802355#file1802355line1444>
> >
> > Is this really a problem? It's something that we see all the time, even
> > when just starting up. Does the fact that the config type is missing from
> > allConfigs signify a problem? If not, we can just not log it.
> >
> > If we want to keep looking it, then why not just find all missing types
> > and print 1 log message with them separated by StringUtils.join(), and then
> > suppresss that one message.
Fair enough, I am thinking of moving this as a Warnign in the DB consistency
checker where user sees it once and can actually autofix it if needed. Opinion?
- Sid
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61833/#review183736
-------
On Aug. 23, 2017, 8:05 p.m., Sid Wagle wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61833/
> ---
>
> (Updated Aug. 23, 2017, 8:05 p.m.)
>
>
> Review request for Ambari, Jonathan Hurley, Myroslav Papirkovskyy, Sumit
> Mohanty, and Sebastian Toader.
>
>
> Bugs: AMBARI-21784
> https://issues.apache.org/jira/browse/AMBARI-21784
>
>
> Repository: ambari
>
>
> Description
> ---
>
> Ambari server log is constantly populated with 'Config inconsistency exists:
> unknown configType=solr-site' even after Solr deletion due to existence of
> config group.
>
> {code}
> 22 Aug 2017 19:00:20,094 ERROR [ambari-hearbeat-monitor] HostImpl:1085 -
> Config inconsistency exists: unknown configType=solr-site
> {code}
>
>
> Diffs
> -
>
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ConfigGroupResourceProvider.java
> 200cf27
>
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
> 3fb45ab
>
> ambari-server/src/main/java/org/apache/ambari/server/state/configgroup/ConfigGroupFactory.java
> 906d948
>
> ambari-server/src/main/java/org/apache/ambari/server/state/configgroup/ConfigGroupImpl.java
> 9058a63
>
> ambari-server/src/main/java/org/apache/ambari/server/state/host/HostImpl.java
> 7282485
>
> ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerTest.java
> 5214f04
>
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ConfigGroupResourceProviderTest.java
> eec379e
>
> ambari-server/src/test/java/org/apache/ambari/server/state/ConfigGroupTest.java
> c70dcc6
>
> ambari-server/src/test/java/org/apache/ambari/server/state/ConfigHelperTest.java
> 21ce2c2
> ambari-server/src/test/java/org/apache/ambari/server/state/ServiceTest.java
> 1ad8cef
>
> ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClusterTest.java
> e8e7206
>
> ambari-server/src/test/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostTest.java
> fc1f334
>
>
> Diff: https://reviews.apache.org/r/61833/diff/3/
>
>
> Testing
> ---
>
> Manually verfied with a 2.5.2 cluster.
> Unit test added.
> All modified unit test pass.
>
>
> Thanks,
>
> Sid Wagle
>
>
Re: Review Request 61875: Oozie server crashes post migration after regen kerberos keytabs and restart all services
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61875/#review183695
---
Ship it!
Ship It!
- Sid Wagle
On Aug. 24, 2017, 1:54 a.m., Robert Levas wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61875/
> ---
>
> (Updated Aug. 24, 2017, 1:54 a.m.)
>
>
> Review request for Ambari, Attila Magyar, Balázs Bence Sári, Di Li, Eugene
> Chekanskiy, Laszlo Puskas, Sebastian Toader, and Sid Wagle.
>
>
> Bugs: AMBARI-21798
> https://issues.apache.org/jira/browse/AMBARI-21798
>
>
> Repository: ambari
>
>
> Description
> ---
>
> This is specifically for the IOP *4.2.0 *to HDP migration
>
> Start with an IOP 4.2.0 cluster with both Oozie and Knox installed, enable
> Kerberos then run thru the migration. Once EU is finalized, regen Kerberos
> keytabs via Ambari web UI and check the checkbox to make Ambari restart all
> services as part of the regen process.
>
> Notice Oozie crashes after a successful restart with following error in the
> oozie.log file
> {noformat}
> 2017-08-23 12:39:48,480 FATAL Services:514 - SERVER[umiak2.fyre.ibm.com]
> E0550: Could not normalize host name [${host}], Could not resolve host
> [${host}], ${host}: Name or service not known
> org.apache.oozie.service.ServiceException: E0550: Could not normalize host
> name [${host}], Could not resolve host [${host}], ${host}: Name or service
> not known
> {noformat}
> This is because Knox kerberos.json in IOP 4.2.0 has oozie proxy setting as
> {{"oozie.service.ProxyUserService.proxyuser.knox.hosts": "${hosts}",}}
>
> In HDP 2.6.2, the same property is now
> {{"oozie.service.ProxyUserService.proxyuser.${knox-env/knox_user}.hosts":
> "${clusterHostInfo/knox_gateway_hosts}"}}, where the ${hosts} placeholder is
> changed and Knox params.py no longer has the logic to calculate it .
>
> *A manual fix* is to change Kerboers config for Knox - update
> {{"oozie.service.ProxyUserService.proxyuser.knox.hosts"}} to have value
> {{"${clusterHostInfo/knox_gateway_hosts}"}} ( done via UI, this will kick
> off regen keytabs and restart services)
>
> # Solution
> Find and remove configuration specifications for the following configurations
> in KNOX/KNOX_GATEWAY since they are invalid due to static "knox" embedded in
> the property name and the values like ${host}:
> - oozie-site/oozie.service.ProxyUserService.proxyuser.knox.groups
> - oozie-site/oozie.service.ProxyUserService.proxyuser.knox.hosts
> - webhcat-site/webhcat.proxyuser.knox.groups
> - webhcat-site/webhcat.proxyuser.knox.hosts
> - core-site/hadoop.proxyuser.knox.groups
> - core-site/hadoop.proxyuser.knox.hosts
> -
> falcon-runtime.properties/*.falcon.service.ProxyUserService.proxyuser.knox.groups
> -
> falcon-runtime.properties/*.falcon.service.ProxyUserService.proxyuser.knox.hosts
>
>
> Diffs
> -
>
>
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog252.java
> b83ba643ec
>
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog252Test.java
> d7df68c122
>
>
> Diff: https://reviews.apache.org/r/61875/diff/1/
>
>
> Testing
> ---
>
> manually tested
>
> # Local test results: PENDING
>
> # Jenkins test results: PENDING
>
>
> Thanks,
>
> Robert Levas
>
>
Review Request 61876: Some services had issue coming up after express upgrade on IOP clusters
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61876/ --- Review request for Ambari, Jonathan Hurley, Robert Levas, and Sumit Mohanty. Bugs: AMBARI-21800 https://issues.apache.org/jira/browse/AMBARI-21800 Repository: ambari Description --- Some services had an issue coming up after express upgrade on IOP clusters. Express upgrade was successful as per Ambari UI. It seems like something is wrong with RCO post-upgrade where the older stack version RCO gets recached. Diffs - ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrder.java 234ef26 Diff: https://reviews.apache.org/r/61876/diff/1/ Testing --- Thanks, Sid Wagle
Review Request 61861: Post Ambari upgrade on a migration cluster Oozie failed to restart when LZO compression is enabled
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61861/ --- Review request for Ambari, Di Li, Jonathan Hurley, Robert Levas, and Sumit Mohanty. Bugs: AMBARI-21794 https://issues.apache.org/jira/browse/AMBARI-21794 Repository: ambari Description --- Restart Oozie fails due lzo package installation failure. The rpm package name is different for IOP stack vs HDP. Diffs - ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/alerts/alert_check_oozie_server.py PRE-CREATION ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/files/oozieSmoke2.sh PRE-CREATION ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/files/prepareOozieHdfsDirectories.sh PRE-CREATION ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/files/wrap_ooziedb.sh PRE-CREATION ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/check_oozie_server_status.py PRE-CREATION ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/oozie.py PRE-CREATION ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/oozie_client.py PRE-CREATION ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/oozie_server.py PRE-CREATION ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/oozie_server_upgrade.py PRE-CREATION ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/oozie_service.py PRE-CREATION ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/params.py PRE-CREATION ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/params_linux.py PRE-CREATION ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/params_windows.py PRE-CREATION ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/service_check.py PRE-CREATION ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/status_params.py PRE-CREATION ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/templates/adminusers.txt.j2 PRE-CREATION ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/templates/oozie-log4j.properties.j2 PRE-CREATION ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/templates/oozie.conf.j2 PRE-CREATION ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/templates/zkmigrator_jaas.conf.j2 PRE-CREATION Diff: https://reviews.apache.org/r/61861/diff/1/ Testing --- Manual verification pending. Thanks, Sid Wagle
Re: Review Request 61856: Auto fix service config mapping issue discovered during ambari-upgrade
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61856/#review183641 --- ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog260.java Lines 160 (patched) <https://reviews.apache.org/r/61856/#comment259703> Why doesn't this check is service exists or not? ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog260.java Lines 168 (patched) <https://reviews.apache.org/r/61856/#comment259704> Is this within a transaction boundary? ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog260Test.java Lines 48 (patched) <https://reviews.apache.org/r/61856/#comment259702> Make sure to run checkstyle on this. - Sid Wagle On Aug. 23, 2017, 6:49 p.m., Vitalyi Brodetskyi wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/61856/ > --- > > (Updated Aug. 23, 2017, 6:49 p.m.) > > > Review request for Ambari, Myroslav Papirkovskyy and Sid Wagle. > > > Bugs: AMBARI-21795 > https://issues.apache.org/jira/browse/AMBARI-21795 > > > Repository: ambari > > > Description > --- > > If user has deleted services, after upgrade db consistency check will throw > warnings about unmapped configs. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java > 40049c0 > > ambari-server/src/main/java/org/apache/ambari/server/orm/entities/ClusterConfigEntity.java > 34f3034 > ambari-server/src/main/java/org/apache/ambari/server/state/ServiceImpl.java > 319bf53 > > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog260.java > 7fc392d > ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql 9cdf8e8 > ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql d616e77 > ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 0530651 > ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 1d16eb6 > ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql da23e95 > ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql e410555 > > ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog260Test.java > 44b5d91 > > > Diff: https://reviews.apache.org/r/61856/diff/2/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Vitalyi Brodetskyi > >
Re: Review Request 61833: Ambari server logs consistency check warning after service delete with ConfigGroup present
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61833/
---
(Updated Aug. 23, 2017, 5:50 p.m.)
Review request for Ambari, Myroslav Papirkovskyy, Sumit Mohanty, and Sebastian
Toader.
Bugs: AMBARI-21784
https://issues.apache.org/jira/browse/AMBARI-21784
Repository: ambari
Description
---
Ambari server log is constantly populated with 'Config inconsistency exists:
unknown configType=solr-site' even after Solr deletion due to existence of
config group.
{code}
22 Aug 2017 19:00:20,094 ERROR [ambari-hearbeat-monitor] HostImpl:1085 - Config
inconsistency exists: unknown configType=solr-site
{code}
Diffs
-
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ConfigGroupResourceProvider.java
200cf27
ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
3fb45ab
ambari-server/src/main/java/org/apache/ambari/server/state/configgroup/ConfigGroupFactory.java
906d948
ambari-server/src/main/java/org/apache/ambari/server/state/configgroup/ConfigGroupImpl.java
9058a63
ambari-server/src/main/java/org/apache/ambari/server/state/host/HostImpl.java
7282485
ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerTest.java
5214f04
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ConfigGroupResourceProviderTest.java
eec379e
ambari-server/src/test/java/org/apache/ambari/server/state/ConfigGroupTest.java
c70dcc6
ambari-server/src/test/java/org/apache/ambari/server/state/ConfigHelperTest.java
21ce2c2
ambari-server/src/test/java/org/apache/ambari/server/state/ServiceTest.java
1ad8cef
ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClusterTest.java
e8e7206
ambari-server/src/test/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostTest.java
fc1f334
Diff: https://reviews.apache.org/r/61833/diff/3/
Testing (updated)
---
Manually verfied with a 2.5.2 cluster.
Unit test added.
All modified unit test pass.
Thanks,
Sid Wagle
Re: Review Request 61833: Ambari server logs consistency check warning after service delete with ConfigGroup present
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61833/
---
(Updated Aug. 23, 2017, 5:10 p.m.)
Review request for Ambari, Myroslav Papirkovskyy, Sumit Mohanty, and Sebastian
Toader.
Changes
---
Minor nit corrected.
Bugs: AMBARI-21784
https://issues.apache.org/jira/browse/AMBARI-21784
Repository: ambari
Description
---
Ambari server log is constantly populated with 'Config inconsistency exists:
unknown configType=solr-site' even after Solr deletion due to existence of
config group.
{code}
22 Aug 2017 19:00:20,094 ERROR [ambari-hearbeat-monitor] HostImpl:1085 - Config
inconsistency exists: unknown configType=solr-site
{code}
Diffs (updated)
-
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ConfigGroupResourceProvider.java
200cf27
ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
3fb45ab
ambari-server/src/main/java/org/apache/ambari/server/state/configgroup/ConfigGroupFactory.java
906d948
ambari-server/src/main/java/org/apache/ambari/server/state/configgroup/ConfigGroupImpl.java
9058a63
ambari-server/src/main/java/org/apache/ambari/server/state/host/HostImpl.java
7282485
ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerTest.java
5214f04
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ConfigGroupResourceProviderTest.java
eec379e
ambari-server/src/test/java/org/apache/ambari/server/state/ConfigGroupTest.java
c70dcc6
ambari-server/src/test/java/org/apache/ambari/server/state/ConfigHelperTest.java
21ce2c2
ambari-server/src/test/java/org/apache/ambari/server/state/ServiceTest.java
1ad8cef
ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClusterTest.java
e8e7206
ambari-server/src/test/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostTest.java
fc1f334
Diff: https://reviews.apache.org/r/61833/diff/3/
Changes: https://reviews.apache.org/r/61833/diff/2-3/
Testing
---
Manually verfied with a 2.5.2 cluster.
Unit test added.
Thanks,
Sid Wagle
Re: Review Request 61833: Ambari server logs consistency check warning after service delete with ConfigGroup present
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61833/
---
(Updated Aug. 23, 2017, 5:08 p.m.)
Review request for Ambari, Myroslav Papirkovskyy, Sumit Mohanty, and Sebastian
Toader.
Changes
---
- Addressed cahnge suggested by Sebastian Toader.
- Removed @Transactional annotation on deleteService(Service service), because
it is calling transactional methods which would just bind to the enclosing
transaction from the callers.
Bugs: AMBARI-21784
https://issues.apache.org/jira/browse/AMBARI-21784
Repository: ambari
Description
---
Ambari server log is constantly populated with 'Config inconsistency exists:
unknown configType=solr-site' even after Solr deletion due to existence of
config group.
{code}
22 Aug 2017 19:00:20,094 ERROR [ambari-hearbeat-monitor] HostImpl:1085 - Config
inconsistency exists: unknown configType=solr-site
{code}
Diffs (updated)
-
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ConfigGroupResourceProvider.java
200cf27
ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
3fb45ab
ambari-server/src/main/java/org/apache/ambari/server/state/configgroup/ConfigGroupFactory.java
906d948
ambari-server/src/main/java/org/apache/ambari/server/state/configgroup/ConfigGroupImpl.java
9058a63
ambari-server/src/main/java/org/apache/ambari/server/state/host/HostImpl.java
7282485
ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerTest.java
5214f04
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ConfigGroupResourceProviderTest.java
eec379e
ambari-server/src/test/java/org/apache/ambari/server/state/ConfigGroupTest.java
c70dcc6
ambari-server/src/test/java/org/apache/ambari/server/state/ConfigHelperTest.java
21ce2c2
ambari-server/src/test/java/org/apache/ambari/server/state/ServiceTest.java
1ad8cef
ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClusterTest.java
e8e7206
ambari-server/src/test/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostTest.java
fc1f334
Diff: https://reviews.apache.org/r/61833/diff/2/
Changes: https://reviews.apache.org/r/61833/diff/1-2/
Testing
---
Manually verfied with a 2.5.2 cluster.
Unit test added.
Thanks,
Sid Wagle
Review Request 61833: Ambari server logs consistency check warning after service delete with ConfigGroup present
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61833/
---
Review request for Ambari, Myroslav Papirkovskyy, Sumit Mohanty, and Sebastian
Toader.
Bugs: AMBARI-21784
https://issues.apache.org/jira/browse/AMBARI-21784
Repository: ambari
Description
---
Ambari server log is constantly populated with 'Config inconsistency exists:
unknown configType=solr-site' even after Solr deletion due to existence of
config group.
{code}
22 Aug 2017 19:00:20,094 ERROR [ambari-hearbeat-monitor] HostImpl:1085 - Config
inconsistency exists: unknown configType=solr-site
{code}
Diffs
-
ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
3fb45ab
ambari-server/src/main/java/org/apache/ambari/server/state/configgroup/ConfigGroupImpl.java
9058a63
ambari-server/src/main/java/org/apache/ambari/server/state/host/HostImpl.java
7282485
ambari-server/src/test/java/org/apache/ambari/server/state/ServiceTest.java
1ad8cef
Diff: https://reviews.apache.org/r/61833/diff/1/
Testing
---
Manually verfied with a 2.5.2 cluster.
Unit test added.
Thanks,
Sid Wagle
Re: Review Request 61582: Not able to start Yarn services after restoring the configs to initial value
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61582/#review183382
---
ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
Lines 1800 (patched)
<https://reviews.apache.org/r/61582/#comment259405>
Fix internal id number.
- Sid Wagle
On Aug. 21, 2017, 8:01 p.m., Vitalyi Brodetskyi wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61582/
> ---
>
> (Updated Aug. 21, 2017, 8:01 p.m.)
>
>
> Review request for Ambari, Myroslav Papirkovskyy, Sumit Mohanty, and Sid
> Wagle.
>
>
> Bugs: AMBARI-21173
> https://issues.apache.org/jira/browse/AMBARI-21173
>
>
> Repository: ambari
>
>
> Description
> ---
>
> Change Yarn-site.xml to some custom configs and restart Yarn
> Restore it back to the origin config; Restart fails
> {Code}
> Traceback (most recent call last):
> File
> "/var/lib/ambari-agent/cache/common-services/YARN/3.0.0.3.0/package/scripts/nodemanager.py",
> line 106, in
> Nodemanager().execute()
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
> line 330, in execute
> method(env)
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
> line 835, in restart
> self.stop(env, upgrade_type=upgrade_type)
> File
> "/var/lib/ambari-agent/cache/common-services/YARN/3.0.0.3.0/package/scripts/nodemanager.py",
> line 45, in stop
> import params
> File
> "/var/lib/ambari-agent/cache/common-services/YARN/3.0.0.3.0/package/scripts/params.py",
> line 29, in
> from params_linux import *
> File
> "/var/lib/ambari-agent/cache/common-services/YARN/3.0.0.3.0/package/scripts/params_linux.py",
> line 39, in
> import status_params
> File
> "/var/lib/ambari-agent/cache/common-services/YARN/3.0.0.3.0/package/scripts/status_params.py",
> line 46, in
> yarn_pid_dir = format("{yarn_pid_dir_prefix}/{yarn_user}")
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/format.py",
> line 95, in format
> return ConfigurationFormatter().format(format_string, args, **result)
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/format.py",
> line 59, in format
> result_protected = self.vformat(format_string, args, all_params)
> File "/usr/lib64/python2.7/string.py", line 549, in vformat
> result = self._vformat(format_string, args, kwargs, used_args, 2)
> File "/usr/lib64/python2.7/string.py", line 582, in _vformat
> result.append(self.format_field(obj, format_spec))
> File "/usr/lib64/python2.7/string.py", line 599, in format_field
> return format(value, format_spec)
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/config_dictionary.py",
> line 73, in __getattr__
> raise Fail("Configuration parameter '" + self.name + "' was not found in
> configurations dictionary!")
> resource_management.core.exceptions.Fail: Configuration parameter 'yarn-env'
> was not found in configurations dictionary!
> {Code}
>
>
> Diffs
> -
>
>
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
> 84b411c
>
>
> Diff: https://reviews.apache.org/r/61582/diff/3/
>
>
> Testing
> ---
>
> mvn clean test
>
>
> Thanks,
>
> Vitalyi Brodetskyi
>
>
