date:20160318

Re: Review Request 45030: Replace 0.0.0.0 with hostname in ams-site/timeline.metrics.service.webapp.address

2016-03-18 Thread Jonathan Hurley


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45030/#review124205
---


Ship it!




Ship It!

- Jonathan Hurley


On March 18, 2016, 11:33 a.m., Dmytro Sen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45030/
> ---
> 
> (Updated March 18, 2016, 11:33 a.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Robert Nettleton, and Sid Wagle.
> 
> 
> Bugs: AMBARI-15480
> https://issues.apache.org/jira/browse/AMBARI-15480
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> We need collector hostname in configs. 
> Replace 0.0.0.0 with hostname in 
> ams-site/timeline.metrics.service.webapp.address
> Implement changes for blueprints, stack_advisor, upgrade catalog
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
>  f5e7578 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog222.java
>  84bb9f3 
>   
> ambari-server/src/main/resources/common-services/ACCUMULO/1.6.1.2.2.0/package/scripts/params.py
>  a21035e 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml
>  99e97f7 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py
>  12c0f25 
>   
> ambari-server/src/main/resources/common-services/FLUME/1.4.0.2.0/package/scripts/params.py
>  29f71a7 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py
>  b32aedf 
>   
> ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/package/scripts/params.py
>  d3a0606 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/params_linux.py
>  ab935a1 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/params.py
>  7153c54 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
> ef82bce 
>   
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/hooks/before-START/scripts/params.py
>  469e709 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
>  68d5755 
>   
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog222Test.java
>  acbf055 
>   ambari-server/src/test/python/stacks/2.0.6/configs/default.json b23d4b9 
>   
> ambari-server/src/test/python/stacks/2.0.6/configs/default_ams_embedded.json 
> 688dbe0 
>   
> ambari-server/src/test/python/stacks/2.0.6/configs/default_hive_non_hdfs.json 
> 6d92182 
>   ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py 
> 921ee23 
>   
> ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade-hdfs-secure.json
>  a43f15a 
>   ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade.json 
> 59cd868 
>   ambari-server/src/test/python/stacks/2.2/configs/ranger-admin-upgrade.json 
> 481b6d4 
>   
> ambari-server/src/test/python/stacks/2.2/configs/ranger-usersync-upgrade.json 
> 200ad23 
>   ambari-server/src/test/python/stacks/2.3/configs/ats_1_5.json 3fe1235 
> 
> Diff: https://reviews.apache.org/r/45030/diff/
> 
> 
> Testing
> ---
> 
> Unit tests passed
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Re: Review Request 45035: Restarting HDFS Before Upgrade Finalizing Does Not Supply the rollingUpgrade Flag

2016-03-18 Thread Nate Cole


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45035/#review124217
---


Ship it!





ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java
 (line 461)


formatting :)


- Nate Cole


On March 18, 2016, 11:20 a.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45035/
> ---
> 
> (Updated March 18, 2016, 11:20 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Nate Cole, and Robert Levas.
> 
> 
> Bugs: AMBARI-15482
> https://issues.apache.org/jira/browse/AMBARI-15482
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> During an upgrade, the HDFS NameNode(s) are restarted with the 
> {{-rollingUpgrade}} flag. However, it's possible to get to the end of an 
> upgrade and decide to "Finalize Later".
> 
> This allows the cluster to run in the upgraded state before committing to the 
> upgrade. Full cluster control is returned via the Ambari web client.
> 
> Administrators can then decide to restart a NameNode. Upon restarting the 
> NameNode, it will produce an error that it was not started with the 
> {{rollingUpgrade}} flag. 
> 
> It seems that as long as an upgrade has not been finalized, the NameNode(s) 
> must be started with the {{rollingUpgrade}} to allow them to function 
> properly. 
> 
> STR:
> - Perform a rolling upgrade from HDP 2.2 to 2.3 (or 2.3 to 2.4); as long as 
> there is a major version change.
> - Before finalization, say "Finalize Later". All services should be up and 
> green.
> - Now restart a NameNode
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/ExecutionCommand.java
>  9ea541e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariActionExecutionHelper.java
>  4ef215c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java
>  303f3a4 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java
>  07061e1 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/UpgradeEntity.java
>  3f1a52b 
>   ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java 
> ed3c772 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
>  1c7ff61 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog222.java
>  84bb9f3 
>   ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql a85202d 
>   ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 9b4810c 
>   ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql cc3d197 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 07c786d 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql 
> ab6dc93 
>   ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 8e91fde 
>   ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 440ca44 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py
>  e4c8c9c 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py
>  02905ec 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py
>  905802f 
>   ambari-web/app/controllers/main/admin/stack_and_upgrade_controller.js 
> 2dceccc 
>   ambari-web/app/utils/ajax/ajax.js 29d0715 
> 
> Diff: https://reviews.apache.org/r/45035/diff/
> 
> 
> Testing
> ---
> 
> Pending...
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Re: Review Request 44986: AMBARI-15474: Listen for changes to auto-start configuration and send them to the agent during heartbeats

2016-03-18 Thread Nate Cole


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44986/#review124220
---




ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java
 (lines 287 - 289)


Use {} format for log statements like so: LOG.info("Recovery configuration 
set to {}", response.getRecoveryConfig());


- Nate Cole


On March 17, 2016, 7 p.m., Nahappan Somasundaram wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44986/
> ---
> 
> (Updated March 17, 2016, 7 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Nate Cole, Sumit Mohanty, and Sid 
> Wagle.
> 
> 
> Bugs: AMBARI-15474
> https://issues.apache.org/jira/browse/AMBARI-15474
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> AMBARI-15474: Listen for changes to auto-start configuration and send them to 
> the agent during heartbeats.
> 
> In HeartbeatHandler::handleHeartBeat(), send the recovery configuration to 
> the agent. Instead of pulling the recovery configuration from the application 
> cache, implement change events and get notified whenever there are changes to 
> the recovery configuration.
> 
> **Changes**:
> 1. Instead of sending the recovery configuration to the agent during every 
> heartbeat (10 seconds interval), send the configuration only when changes are 
> detected.
> 2. Used AmbariEventPublisher to publish the changes. RecoveryConfigHelper is 
> the subscriber which listens to changes to the configuration.
> 3. RecoveryConfigHelper maintains a map of Cluster-->Host-->Timestamp which 
> is the last updated timestamp of the recovery configuration. This timestamp 
> is used as a token and is sent to the agent with the configuration. The agent 
> provides this timestamp during each heartbeat. If the timestamp from the 
> heartbeat is the same as the one on the server side, no configuration is sent 
> to the agent.
> 4. Whenever changes are detected, the timestamp is invalidated. During the 
> next heartbeat, the recovery configuration is read from the DB and sent to 
> the agent along with the latest timestamp.
> 5. Added new unit tests to test each configuration change event.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/Controller.py 
> c1c16ac97cca0d2677da2e26eee7b4949a4bf15c 
>   ambari-agent/src/main/python/ambari_agent/Heartbeat.py 
> b28644c834a2387d2fb0ad17d104224e830a0245 
>   ambari-agent/src/main/python/ambari_agent/RecoveryManager.py 
> ed537ca6928ce376c5f3e906f7a47c3f1919e309 
>   ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeat.java 
> 1430aa20cf68db071be85c4ad2ebb9acdaccecc0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java
>  3a80803724bcb6ca9e4ec875aa17c9ac1c66fbe8 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatResponse.java
>  617b04b5b8768ba28a3c8ecb6e5e00601f153396 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/RecoveryConfig.java
>  c2c18462c586292caaf40bdb6a25a8fe9a39d76c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/RecoveryConfigHelper.java
>  92a622117c71ddfc3bd2562c04ee525491bd6ffd 
>   
> ambari-server/src/main/java/org/apache/ambari/server/events/AmbariEvent.java 
> 78731854fe80c3d6087a73ebf00b0ffe9e287354 
>   
> ambari-server/src/main/java/org/apache/ambari/server/events/ClusterConfigChangedEvent.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/events/ServiceComponentInstalledEvent.java
>  2ce01236b3bb1ab4934fe456e5669fa474987426 
>   
> ambari-server/src/main/java/org/apache/ambari/server/events/ServiceComponentRecoveryChangeEvent.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/events/ServiceComponentUninstalledEvent.java
>  19712cdb18592816f82898cd7aa08a3600c7e1f4 
>   ambari-server/src/main/java/org/apache/ambari/server/state/ConfigImpl.java 
> 2cc3d0013140c9c7d3453d7c5e05c3aa02bc5794 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/ServiceComponentImpl.java
>  4be1c21bf41416681be836716ce38305cc3f287e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java
>  3a0075e3a3c90088ca6bd2d9eca48ef50db9fdb3 
>   
> ambari-server/src/test/java/org/apache/ambari/server/agent/TestHeartbeatHandler.java
>  b1f94e38f97550123dd366984438b0bbf9c2826c 
>   
> ambari-server/src/test/java/org/apache/ambari/server/configuration/RecoveryConfigHelperTest.java
>  c86b95a7cd10b0720f0a3b9676f65a8bd68dd6c9 
>   
> ambari-server/src/test/java/org/apache/ambari/server/events/listeners/upgrade/HostVersionOutOfSyncListenerTest.java
>

Re: Review Request 44983: Service version display should be based on Version Definition

2016-03-18 Thread Jonathan Hurley


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44983/#review124177
---


Ship it!




Ship It!

- Jonathan Hurley


On March 17, 2016, 5:53 p.m., Nate Cole wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44983/
> ---
> 
> (Updated March 17, 2016, 5:53 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez and Jonathan Hurley.
> 
> 
> Bugs: AMBARI-15473
> https://issues.apache.org/jira/browse/AMBARI-15473
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add {{stack_services}} to the repository_version endpoint, similar to the 
> {{services}} attribute that lists the available services.
> 
> This will return a list of all stack services as they pertain to the Version 
> Definition File.  This is required to show the correct versions that are 
> available for a stack based on the repo_version.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RepositoryVersionResourceProvider.java
>  c298e0a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/repository/ManifestServiceInfo.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/repository/VersionDefinitionXml.java
>  93ac767 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/repository/VersionDefinitionTest.java
>  f2939c7 
>   ambari-server/src/test/resources/version_definition_test.xml 69ea581 
> 
> Diff: https://reviews.apache.org/r/44983/diff/
> 
> 
> Testing
> ---
> 
> Manual.  Automated pending.
> 
> 
> Thanks,
> 
> Nate Cole
> 
>

Review Request 44987: Editing tmp dir from single to multiple dir doesn't trigger the backend change

2016-03-18 Thread jun aoki


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44987/
---

Review request for Ambari.


Bugs: AMBARI-15475
https://issues.apache.org/jira/browse/AMBARI-15475


Repository: ambari


Description
---

Modified "HAWQ Segment Temp Directory” from one [/d1/tmp/hawqsegment] to many 
[/d1/tmp/hawqsegment,/d2/tmp/hawqsegment,/d3/tmp/hawqsegment,/d4/tmp/hawqsegment,/d5/tmp/hawqsegment]
and the change is not propagated to segments and the folders are not created.


Diffs
-

  
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/common.py
 d62332d 
  
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/hawqsegment.py
 0a597b6 
  
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/master_helper.py
 9482b35 
  
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/params.py
 1bdc5aa 

Diff: https://reviews.apache.org/r/44987/diff/


Testing
---

manually tested.


Thanks,

jun aoki

Re: Review Request 45054: AMBARI-15443:Make Host bulk command menu item list stack driven instead of a hardcoded list in UI code

2016-03-18 Thread Zhe (Joe) Wang


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45054/#review124335
---



There are some assumptions about certain operations for certain slave 
components. So please verify the ops are valid before add new components to 
this list. (Current ones are ok)
One example is that adding NFS_GATEWAY requires HDFS_CLIENT on the same host. 
But we don't do dependency check here when add components as it's not needed 
for current components in the list. But adding NFS_GATEWAY to this list may 
cause problem.

- Zhe (Joe) Wang


On March 18, 2016, 9:46 p.m., Di Li wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45054/
> ---
> 
> (Updated March 18, 2016, 9:46 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jaimin Jetly, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-15443
> https://issues.apache.org/jira/browse/AMBARI-15443
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> The items described here are the ones on the Hosts tab, in the Actions 
> drop-down list where the UI shows entries such as All Hosts. If user mouses 
> over the All Hosts, it shows a sub-list including Hosts and slave components.
> 
> The slave component items are hardcoded in hosts_table_menu_view.js as shown 
> below. This jira is to put this info into each service's metainfo.xml so that 
> the slave component items can be stack driven.
> 
> components: function () {
> var serviceNames = App.Service.find().mapProperty('serviceName');
> var menuItems = [
> O.create(
> { serviceName: 'HDFS', componentName: 'DATANODE', masterComponentName: 
> 'NAMENODE', componentNameFormatted: 
> Em.I18n.t('dashboard.services.hdfs.datanodes') }
> 
> ),
> O.create(
> { serviceName: 'YARN', componentName: 'NODEMANAGER', masterComponentName: 
> 'RESOURCEMANAGER', componentNameFormatted: 
> Em.I18n.t('dashboard.services.yarn.nodeManagers') }
> 
> ),
> O.create(
> { serviceName: 'HBASE', componentName: 'HBASE_REGIONSERVER', 
> masterComponentName: 'HBASE_MASTER', componentNameFormatted: 
> Em.I18n.t('dashboard.services.hbase.regionServers') }
> 
> ),
> O.create(
> { serviceName: 'STORM', componentName: 'SUPERVISOR', masterComponentName: 
> 'SUPERVISOR', componentNameFormatted: 
> Em.I18n.t('dashboard.services.storm.supervisors') }
> 
> )];
> 
> return menuItems.filter(function (item)
> { return serviceNames.contains(item.serviceName); }
> 
> );
> }.property(),
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/StackServiceComponentResponse.java
>  cfd4e7b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackServiceComponentResourceProvider.java
>  70945ba 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ComponentModule.java
>  a122dc6 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/BulkCommandDefinition.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/ComponentInfo.java 
> 770ee5c 
>   ambari-server/src/main/resources/common-services/HAWQ/2.0.0/metainfo.xml 
> e35b7d8 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/metainfo.xml
>  057e126 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml 
> 75d3bea 
>   ambari-server/src/main/resources/common-services/PXF/3.0.0/metainfo.xml 
> afe27ec 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/metainfo.xml 
> 804374a 
>   
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/metainfo.xml 
> 0f71585 
>   ambari-server/src/main/resources/properties.json 01c15f2 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/ComponentModuleTest.java
>  2f84f04 
>   ambari-web/app/mappers/stack_service_mapper.js 8a65055 
>   ambari-web/app/models/stack_service_component.js 26ff1b8 
>   ambari-web/app/views/main/host/hosts_table_menu_view.js e75b643 
>   ambari-web/test/mappers/stack_service_mapper_test.js 4bc36fe 
> 
> Diff: https://reviews.apache.org/r/45054/diff/
> 
> 
> Testing
> ---
> 
> unit test
> patch a trunk cluster with code change, verify the host bulk command list 
> shown for DataNode, HBase Region server, storm supervisor, etc.
> 
> 
> Thanks,
> 
> Di Li
> 
>

Re: Review Request 45030: Replace 0.0.0.0 with hostname in ams-site/timeline.metrics.service.webapp.address

2016-03-18 Thread Robert Nettleton


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45030/#review124190
---


Ship it!




Ship It!

- Robert Nettleton


On March 18, 2016, 1:45 p.m., Dmytro Sen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45030/
> ---
> 
> (Updated March 18, 2016, 1:45 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Robert Nettleton, and Sid Wagle.
> 
> 
> Bugs: AMBARI-15480
> https://issues.apache.org/jira/browse/AMBARI-15480
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> We need collector hostname in configs. 
> Replace 0.0.0.0 with hostname in 
> ams-site/timeline.metrics.service.webapp.address
> Implement changes for blueprints, stack_advisor, upgrade catalog
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
>  e73e38f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog222.java
>  e10585e 
>   
> ambari-server/src/main/resources/common-services/ACCUMULO/1.6.1.2.2.0/package/scripts/params.py
>  3b8505b 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml
>  45c1811 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py
>  12c0f25 
>   
> ambari-server/src/main/resources/common-services/FLUME/1.4.0.2.0/package/scripts/params.py
>  5ec879c 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py
>  b517eeb 
>   
> ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/package/scripts/params.py
>  f75d578 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/params_linux.py
>  a022282 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/params.py
>  7a023cf 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
> ee2007f 
>   
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/hooks/before-START/scripts/params.py
>  469e709 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
>  ef5dc76 
>   
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog222Test.java
>  859513b 
>   ambari-server/src/test/python/stacks/2.0.6/configs/default.json fe92b5b 
>   
> ambari-server/src/test/python/stacks/2.0.6/configs/default_ams_embedded.json 
> 688dbe0 
>   
> ambari-server/src/test/python/stacks/2.0.6/configs/default_hive_non_hdfs.json 
> 4648ffc 
>   ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py 
> 9b2da28 
>   
> ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade-hdfs-secure.json
>  68afb68 
>   ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade.json 
> 79c5580 
>   ambari-server/src/test/python/stacks/2.2/configs/ranger-admin-upgrade.json 
> a7057db 
>   
> ambari-server/src/test/python/stacks/2.2/configs/ranger-usersync-upgrade.json 
> 390f9ce 
>   ambari-server/src/test/python/stacks/2.3/configs/ats_1_5.json 3fe1235 
> 
> Diff: https://reviews.apache.org/r/45030/diff/
> 
> 
> Testing
> ---
> 
> Unit tests passed
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Re: Review Request 44931: Use Version Definition value for package-version when installing

2016-03-18 Thread Alejandro Fernandez


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44931/#review124064
---


Ship it!




Ship It!

- Alejandro Fernandez


On March 17, 2016, 12:51 p.m., Nate Cole wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44931/
> ---
> 
> (Updated March 17, 2016, 12:51 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez and Jonathan Hurley.
> 
> 
> Bugs: AMBARI-15447
> https://issues.apache.org/jira/browse/AMBARI-15447
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> The version defintion file defines a package version that should be used when 
> installing packages.  The current calculation cannot be considered reliable, 
> but should stay as a contingency for when NOT using a version definition.
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/script/script.py 
> a8098a0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/ExecutionCommand.java
>  402a338 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  e32e2f9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProvider.java
>  07e62b3 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostStackVersionResourceProvider.java
>  1cd9c0a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RepositoryVersionEntity.java
>  3398709 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/repository/Release.java
>  6bcedf5 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProviderTest.java
>  dea83a1 
>   ambari-server/src/test/python/custom_actions/TestInstallPackages.py f022c80 
>   ambari-server/src/test/resources/hbase_version_test.xml 9df07ed 
> 
> Diff: https://reviews.apache.org/r/44931/diff/
> 
> 
> Testing
> ---
> 
> Manual.  Automated:
> 
> Results :
> 
> Tests run: 3965, Failures: 0, Errors: 0, Skipped: 33
> 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 1:10:59.814s
> [INFO] Finished at: Thu Mar 17 08:26:54 EDT 2016
> [INFO] Final Memory: 38M/566M
> [INFO] 
> 
> 
> 
> Thanks,
> 
> Nate Cole
> 
>

Re: Review Request 44931: Use Version Definition value for package-version when installing

2016-03-18 Thread Nate Cole


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44931/
---

(Updated March 17, 2016, 8:51 a.m.)


Review request for Ambari, Alejandro Fernandez and Jonathan Hurley.


Bugs: AMBARI-15447
https://issues.apache.org/jira/browse/AMBARI-15447


Repository: ambari


Description
---

The version defintion file defines a package version that should be used when 
installing packages.  The current calculation cannot be considered reliable, 
but should stay as a contingency for when NOT using a version definition.


Diffs
-

  ambari-common/src/main/python/resource_management/libraries/script/script.py 
a8098a0 
  
ambari-server/src/main/java/org/apache/ambari/server/agent/ExecutionCommand.java
 402a338 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
 e32e2f9 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProvider.java
 07e62b3 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostStackVersionResourceProvider.java
 1cd9c0a 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RepositoryVersionEntity.java
 3398709 
  
ambari-server/src/main/java/org/apache/ambari/server/state/repository/Release.java
 6bcedf5 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProviderTest.java
 dea83a1 
  ambari-server/src/test/python/custom_actions/TestInstallPackages.py f022c80 
  ambari-server/src/test/resources/hbase_version_test.xml 9df07ed 

Diff: https://reviews.apache.org/r/44931/diff/


Testing (updated)
---

Manual.  Automated:

Results :

Tests run: 3965, Failures: 0, Errors: 0, Skipped: 33

[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 1:10:59.814s
[INFO] Finished at: Thu Mar 17 08:26:54 EDT 2016
[INFO] Final Memory: 38M/566M
[INFO] 


Thanks,

Nate Cole

Re: Review Request 44956: Topology host info is not cleared when a host is removed

2016-03-18 Thread Daniel Gergely


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44956/
---

(Updated márc. 18, 2016, 9:27 de)


Review request for Ambari, Alejandro Fernandez, Laszlo Puskas, Oliver Szabo, 
Sandor Magyari, Sumit Mohanty, and Sebastian Toader.


Bugs: AMBARI-15457
https://issues.apache.org/jira/browse/AMBARI-15457


Repository: ambari


Description
---

When a host is removed, a record is left in the topology_host_info table. As a 
result re-adding the same host is not possible, due to invalid topology.
Host removal should remove the corresponding record from topology_host_info as 
well.

A new column with a foreign key to hosts table is added. In addition 
ClusterTopologyImpl was statefull and hostGroupInfoMap was not updated when a 
host was removed.


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostResourceProvider.java
 6101ee8 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/dao/TopologyHostInfoDAO.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/TopologyHostInfoEntity.java
 a4f251a 
  
ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClustersImpl.java
 46dd49d 
  
ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterTopology.java
 4e178c0 
  
ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterTopologyImpl.java
 003539c 
  
ambari-server/src/main/java/org/apache/ambari/server/topology/HostGroupInfo.java
 4847a11 
  
ambari-server/src/main/java/org/apache/ambari/server/topology/PersistedState.java
 dbf6735 
  
ambari-server/src/main/java/org/apache/ambari/server/topology/PersistedStateImpl.java
 b878955 
  
ambari-server/src/main/java/org/apache/ambari/server/topology/TopologyManager.java
 41f3150 
  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog222.java
 8267d5d 
  ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql dee397f 
  ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 253b5d7 
  ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 832d1bc 
  ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql 
6c8a09d 
  ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql a6acb74 
  ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 2957c5c 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerTest.java
 3e43ef1 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/HostResourceProviderTest.java
 69c570e 

Diff: https://reviews.apache.org/r/44956/diff/


Testing
---

Tests ran without failures. (2016-03-17 13:30)


Thanks,

Daniel Gergely

Re: Review Request 44886: AMBARI-15437 : Handle changing symlinks of ranger admin binaries for upgrade scenarios

2016-03-18 Thread Jonathan Hurley


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44886/#review123860
---




ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py
 (lines 126 - 129)


Do you need to:

- Check for the existance of these links first before removing/creating 
them?
- Check a specific version of the stack that you are upgrading from/to?


- Jonathan Hurley


On March 16, 2016, 5:15 a.m., Gautam Borad wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44886/
> ---
> 
> (Updated March 16, 2016, 5:15 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jaimin Jetly, Jonathan 
> Hurley, Mahadev Konar, Sumit Mohanty, Selvamohan Neethiraj, and Velmurugan 
> Periasamy.
> 
> 
> Bugs: AMBARI-15437
> https://issues.apache.org/jira/browse/AMBARI-15437
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> For Ambari stacks for Ranger: 
> We need to take care of handling symlinks to point to current version of HDP, 
> to avoid confusion.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py
>  6394f51 
> 
> Diff: https://reviews.apache.org/r/44886/diff/
> 
> 
> Testing
> ---
> 
> Tested EU and RU with Ranger installed.
> Testing steps:
> 1) Installed Ranger with HDP-2.2.x
> 2) Upgrade Ranger to HDP-2.3.x
> 3) Again Upgrade to HDP-2.4.x
> 
> Ran 277 tests in 7.293s
> 
> OK
> --
> Total run:941
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Gautam Borad
> 
>

Re: Review Request 44835: AMBARI-15420: Refactor resource_management library

2016-03-18 Thread Alejandro Fernandez


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44835/#review124302
---


Ship it!




Ship It!

- Alejandro Fernandez


On March 18, 2016, 8:57 p.m., Jayush Luniya wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44835/
> ---
> 
> (Updated March 18, 2016, 8:57 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Juanjo  
> Marron, Nate Cole, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-15420
> https://issues.apache.org/jira/browse/AMBARI-15420
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Refactor resource_management library, to generalize the library and remove 
> HDP-specific hardcodings
> 
> Specifically,
> 1. Make stack-selector, conf-selector tools stack-driven instead of 
> hardcoding hdp-select and conf-select
> 2. Make stack-root stack-driven instead of hardcoding /usr/hdp
> 3. Make copy_tarball mappings stack-driven 
> 4. Make PACKAGE_DIRS mapping in conf_select use stack-root instead of 
> hardcoding the "/usr/hdp"
> 
> In addition, also added a feature in the stack processing engine to support 
> properties values to be defined a external property file (See 
> tarball_map.json, stack_tools.json in patch)
> 
> Three config properties are added
> 1. cluster-env/stack_root
> 2. cluster-env/stack_tools
> 3. cluster-env/tarball_map
> Corresponding helper functions get_stack_root(), get_stack_tool(), 
> get_tarball_map() are added in script.py, which will set the defaults if 
> these config properties are not defined (ambari-server upgrade scenario needs 
> to be addressed to add these config properties on upgrade). These helper 
> functions are used to remove hardcodings in resource_management library.
> 
> 
> Remaining HDP-specific logic in resource_management library
> 
> 1. conf_select::_valid()
> 2. conf_select::get_hadoop_conf_dir()
> 3. list_ambari_managed_repos::repository_names
> 4. version_select_util::get_component_version()
> 5. script::get_stack_version()
> 6. script::should_expose_component_version()
> 7. get_lzo_packages::get_lzo_packages()
> 
> Refactoring this HDP-specific logic would require "Stack Featurization" 
> (AMBARI-13364) to be in place.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/HostCheckReportFileHandler.py 
> ee7db0a 
>   ambari-common/src/main/python/resource_management/core/utils.py 71d0008 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py
>  770595f 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py
>  647b8b6 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/default.py
>  23383dc 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/dynamic_variable_interpretation.py
>  a20b03c 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/get_stack_version.py
>  f2e6567 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/install_windows_msi.py
>  f1cd9cb 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/repo_version_history.py
>  d585dea 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/setup_ranger_plugin.py
>  4d9d8a4 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/setup_ranger_plugin_xml.py
>  2ccc0c6 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/stack_select.py
>  4a8eeb9 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/stack_tools.py
>  PRE-CREATION 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/version_select_util.py
>  95c5cba 
>   
> ambari-common/src/main/python/resource_management/libraries/script/script.py 
> 1b0116e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java
>  4be4049 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ConfigurationDirectory.java
>  7f21aaa 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java 
> 0c7faea 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackDefinitionDirectory.java
>  c739211 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
> 7d934bb 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/PropertyInfo.java 
> e7c9c27 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/ValueAttributesInfo.java
>  3f7f756 
>   ambari-server/src/main/java/org/apache/ambari/server/utils/JsonUtils.java 
> PRE-CREATION 
>   
>

Review Request 45056: Blueprint install using config_recommendation_strategy is not functional

2016-03-18 Thread Shantanu Mundkur


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45056/
---

Review request for Ambari, Oliver Szabo and Robert Levas.


Bugs: AMBARI-15454
https://issues.apache.org/jira/browse/AMBARI-15454


Repository: ambari


Description
---

hBlueprint install using config_recommendation strategy seems to hang for a 
long time (couple of hours?) and ends up logging exceptions continually to 
ambari-server.log. At the same time many hundreds of directories are seen 
getting created under /var/run/ambari-server/stack-recommendations (I have seen 
above 800-900). If you keep it running eventually the cluster install seems to 
start but fails miserably at least during the start and some of it makes 
obvious that configuration recommendations were NOT applied. You see errors 
during startup that hint that options used for JVM (For Datanode etc) were 
unreasonable.

Note that both the blueprint and cluster templates used empty configurations. 
Example:

.
.
   "configurations" : [],
   "host_groups": [
{
 "name": "host-group-1",
 "configurations" : [],
 "cardinality" : "1",
 "components": [
  { "name": "APP_TIMELINE_SERVER" },
  { "name": "DATANODE" },
  { "name": "FALCON_CLIENT" },
  { "name": "FALCON_SERVER" },
  { "name": "FLUME_HANDLER" },
  { "name": "HBASE_CLIENT" },
  { "name": "HBASE_MASTER" },
  { "name": "HBASE_REGIONSERVER" },
.
.
.

Teh cluster template was:
{ 
"blueprint": "1node",
"config_recommendation_strategy" : "ONLY_STACK_DEFAULTS_APPLY",
"default_password": "myPassword1",
"host_groups": [
{
"name": "host-group-1",
"hosts": [
{
"fqdn": "mynode.ibm.com"
}
]
}
] 
}


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
 f5e7578 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
 68d5755 

Diff: https://reviews.apache.org/r/45056/diff/


Testing
---

1) ambari-server unit tests

Results :

Tests run: 3968, Failures: 0, Errors: 0, Skipped: 33
2) Added unit test to 
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
 
3) Deployed clusters 1-5 nodes specifying "config_recommendation_strategy" : 
"ONLY_STACK_DEFAULTS_APPLY"


Thanks,

Shantanu Mundkur

Re: Review Request 44933: Apply the stack featurization prototype detailed on AMBARI-13364 to ZOOKEEPER service.

2016-03-18 Thread Juanjo Marron

> On March 18, 2016, 9:25 p.m., Jayush Luniya wrote:
> > ambari-common/src/main/python/resource_management/libraries/functions/stack_fetaures.py,
> >  line 1
> > 
> >
> > Minor, fix typo in file name

OK.Thanks. I'll solve it in the next patch replacing this one

> On March 18, 2016, 9:25 p.m., Jayush Luniya wrote:
> > ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5.2.0/package/scripts/params_linux.py,
> >  line 53
> > 
> >
> > Will things might work out in regular install, I think there is going 
> > to be a problem here during upgrades.
> > 
> > Lets say we are doing EU from HDP-2.1 to HDP-2.2
> > the command.json file for the command will have 
> > hostLevelParams/current_version = 2.1.0.0-1234
> > hostLevelParams/version = 2.2.0.0-5678
> > commandParams/stack_version = 2.2
> > configurations/cluster-env/stack_features = "snappy" (HDP-2.1 version 
> > of stack_features)
> > 
> > So the check will fail.
> > 
> > @Alejandro is my understanding correct? I can try out the upgrade to 
> > verify too. 
> > 
> > BTW, @Juanjo Script.is_stack_greater* functions use 
> > commandParams/stack_version whereas you are changing it to current_version
> 
> Jayush Luniya wrote:
> @Juanjo, 
> Alejandro and I had a chat on this. Given the upgrade scenario, it would 
> be best to have a single stack_features list for a stack and maintain a 
> min_version and max_version for the feature instead of breaking down the list 
> of stack features for each stack version. That way we will not run into 
> issues with upgrades where we would sometimes need the upgradeFromVersion 
> stack_features list and sometimes we would need the upgradeToVersion 
> stack_features list.

@Jayush,
@Alejandro
So If I understand properly, the idea is to create a xml file, let's say 
stack_features.xml  under 
ambari-server/src/main/resources/stacks/HDP/configuration,  for example. It 
will be common to all the HDP stacks.
Inside the xml I  can add  the json file content and read it using the new 
feature to support external properties file whicj Jayush is creating in 
https://reviews.apache.org/r/44835. 

stack_features

List of features supported by the stack
VALUE_FROM_PROPERTY_FILE

stack_features.json
json

And stack_fetaures.json will be:

{
"stack_features": [
{ "name": "snappy", "description" : "Snappy compressor/decompressor 
support" , "min_version" : "2.0.0.0" , "max_version" : "2.2.0.0" }
,
{ "name": "upgrade", "description" : "Upgrade support", 
"min_version" : "2.2.0.0" }
,
{ "name": "ranger", "description" : "Ranger Service support", 
"min_version" : "2.2.0.0" }
]
}

This stack_features.xml file will be read from params.py and stack_features.py 
(new resource_management script) will make sure the stack_version for a 
specific feature is in the range [min_version - max_version]. Ill let 
max_version as optional argument since the re will be features working from 
min_version until now and the next versions.

Also, regarding the use of commandParams/stack_version. I would need to declare 
it? it is needed? I mean is it diff from stack_version_unformatted, 
stack_version_formatted , current_version or version that are already defined?

> On March 18, 2016, 9:25 p.m., Jayush Luniya wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.0.6/configuration/cluster-env.xml,
> >  line 25
> > 
> >
> > FYI, in https://reviews.apache.org/r/44835/ I added a feature to 
> > support external properties file for complete structures like these. So you 
> > can create a json file at HDP/2.0.6/properties/stack_features.json and 
> > reference it in the config property xml file. I will be committing those 
> > changes soon.

I saw the example of stack_tools.json in https://reviews.apache.org/r/44835/.
It would work even if the stack_fetaures.json file is defined out of the stack, 
under HDP/configuration/stack_fetautes for example?

- Juanjo

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44933/#review124285
---

On March 16, 2016, 9:24 p.m., Juanjo  Marron wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44933/
> ---
> 
> (Updated March 16, 2016, 9:24 p.m.)
> 
> 
> Review request for Ambari,

Re: Review Request 44958: Increase Ambari Server Perm gen default value

2016-03-18 Thread Sandor Magyari


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44958/#review124174
---


Ship it!




Ship It!

- Sandor Magyari


On March 18, 2016, 8:39 a.m., Daniel Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44958/
> ---
> 
> (Updated March 18, 2016, 8:39 a.m.)
> 
> 
> Review request for Ambari, Laszlo Puskas, Oliver Szabo, Robert Levas, Sandor 
> Magyari, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-15465
> https://issues.apache.org/jira/browse/AMBARI-15465
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Increased permGen size to 128m, also added **-XX:+CMSPermGenSweepingEnabled** 
> and **-XX:+CMSClassUnloadingEnabled** to allow jvm to keep permGen space 
> clean.
> 
> 
> Diffs
> -
> 
>   ambari-server/conf/unix/ambari-env.sh e9fdcdd 
>   ambari-server/src/main/python/ambari_server_main.py 00e084b 
> 
> Diff: https://reviews.apache.org/r/44958/diff/
> 
> 
> Testing
> ---
> 
> Manual testing:
> [root@c6401 sbin]# ps -ef | grep ambari-server
> root 20769 1 27 14:47 pts/000:00:24 
> //usr/jdk64/jdk1.8.0_60/bin/java -server -XX:NewRatio=3 
> -XX:+UseConcMarkSweepGC -XX:-UseGCOverheadLimit 
> -XX:CMSInitiatingOccupancyFraction=60 **-XX:+CMSPermGenSweepingEnabled 
> -XX:+CMSClassUnloadingEnabled** -Dsun.zip.disableMemoryMapping=true -Xms512m 
> -Xmx2048m **-XX:MaxPermSize=128m** 
> -Djava.security.auth.login.config=//etc/ambari-server/conf/krb5JAASLogin.conf 
> -Djava.security.krb5.conf=/etc/krb5.conf 
> -Djavax.security.auth.useSubjectCredsOnly=false -Xms512m -Xmx2048m 
> -Djava.security.auth.login.config=//etc/ambari-server/conf/krb5JAASLogin.conf 
> -Djava.security.krb5.conf=/etc/krb5.conf 
> -Djavax.security.auth.useSubjectCredsOnly=false -cp 
> //etc/ambari-server/conf:/usr/lib/ambari-server/*:/usr/share/java/postgresql-jdbc.jar
>  org.apache.ambari.server.controller.AmbariServer
> 
> 
> Thanks,
> 
> Daniel Gergely
> 
>

Re: Review Request 44969: After upgrading from Ambari 2.1.2 to 2.2.1, HDP from 2.3.0 to 2.3.4 , user is getting 'Consistency Check Failed' error while trying to save configuration in Ranger, Hive and

2016-03-18 Thread Sebastian Toader


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44969/#review124156
---




ambari-server/src/main/resources/stacks/HDP/2.2/services/stack_advisor.py (line 
668)


According to 
http://hbase.apache.org/0.94/book/hbase.accesscontrol.configuration.html when 
authorization is enabled 
'org.apache.hadoop.hbase.security.access.AccessController' is added both to 
'hbase.coprocessor.master.classes' and 'hbase.coprocessor.region.classes' so I 
think when authorization is disabled it should be removed from both.

Split this 'if' into two separate ifs one checking 
'hbase.coprocessor.master.classes' and removing from 
'org.apache.hadoop.hbase.security.access.AccessController' from it and 
similarly another one for 'hbase.coprocessor.region.classes' removing 
'org.apache.hadoop.hbase.security.access.AccessController' from it.


Please perform manual testing of HBase authorisation enable/disbale and verify 
if HBase is still operational.

Ensure that 
ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py unit test 
covers this test case.

- Sebastian Toader


On March 17, 2016, 6:50 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44969/
> ---
> 
> (Updated March 17, 2016, 6:50 p.m.)
> 
> 
> Review request for Ambari, Jaimin Jetly and Sebastian Toader.
> 
> 
> Bugs: AMBARI-15469
> https://issues.apache.org/jira/browse/AMBARI-15469
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> PROBLEM:After upgrading from Ambari 2.1.2 to 2.2.1, HDP from 2.3.0 to 2.3.4 , 
> user is getting 'Consistency Check Failed' error while trying to save 
> configuration in Ranger, Hive and HBase
> 
> Please find the attached screenshot, and ambari-server log for more 
> information.
> 
> {code}
> Error occured in stack advisor.
> Error details: list.remove(x): x not in list
> 14 Mar 2016 14:31:26,241  INFO [qtp-ambari-client-23360] 
> StackAdvisorRunner:71 - advisor script stderr: Traceback (most recent 
> call last):
> File "/var/lib/ambari-server/resources/scripts/stack_advisor.py", line 158, 
> in 
> main(sys.argv)
> File "/var/lib/ambari-server/resources/scripts/stack_advisor.py", line 109, 
> in main
> result = stackAdvisor.recommendConfigurations(services, hosts)
> File "/var/lib/ambari-server/resources/scripts/../stacks/stack_advisor.py", 
> line 569, in recommendConfigurations
> calculation(configurations, clusterSummary, services, hosts)
> File 
> "/var/lib/ambari-server/resources/scripts/./../stacks/HDP/2.3/services/stack_advisor.py",
>  line 239, in recommendHBASEConfigurations
> super(HDP23StackAdvisor, self).recommendHBASEConfigurations(configurations, 
> clusterData, services, hosts)
> File 
> "/var/lib/ambari-server/resources/scripts/./../stacks/HDP/2.2/services/stack_advisor.py",
>  line 665, in recommendHBASEConfigurations
> hbaseCoProcessorConfigs['hbase.coprocessor.master.classes'].remove('org.apache.hadoop.hbase.security.access.AccessController')
> ValueError: list.remove(x): x not in list
> {code}
> 
> Looks like property 'hbase.coprocessor.master.classes' is empty. The 
> workaround is by putting value 
> org.apache.hadoop.hbase.security.access.AccessController there.
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/stacks/HDP/2.2/services/stack_advisor.py 
> 5e23b8a 
> 
> Diff: https://reviews.apache.org/r/44969/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Re: Review Request 44956: Topology host info is not cleared when a host is removed

2016-03-18 Thread Sandor Magyari


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44956/#review124243
---


Ship it!




Ship It!

- Sandor Magyari


On March 18, 2016, 9:27 a.m., Daniel Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44956/
> ---
> 
> (Updated March 18, 2016, 9:27 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Laszlo Puskas, Oliver Szabo, 
> Sandor Magyari, Sumit Mohanty, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-15457
> https://issues.apache.org/jira/browse/AMBARI-15457
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When a host is removed, a record is left in the topology_host_info table. As 
> a result re-adding the same host is not possible, due to invalid topology.
> Host removal should remove the corresponding record from topology_host_info 
> as well.
> 
> A new column with a foreign key to hosts table is added. In addition 
> ClusterTopologyImpl was statefull and hostGroupInfoMap was not updated when a 
> host was removed.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostResourceProvider.java
>  6101ee8 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/TopologyHostInfoDAO.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/TopologyHostInfoEntity.java
>  a4f251a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClustersImpl.java
>  46dd49d 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterTopology.java
>  4e178c0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterTopologyImpl.java
>  003539c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/HostGroupInfo.java
>  4847a11 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/PersistedState.java
>  dbf6735 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/PersistedStateImpl.java
>  b878955 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/TopologyManager.java
>  41f3150 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog222.java
>  8267d5d 
>   ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql dee397f 
>   ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 253b5d7 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 832d1bc 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql 
> 6c8a09d 
>   ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql a6acb74 
>   ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 2957c5c 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerTest.java
>  3e43ef1 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/HostResourceProviderTest.java
>  69c570e 
> 
> Diff: https://reviews.apache.org/r/44956/diff/
> 
> 
> Testing
> ---
> 
> Tests ran without failures. (2016-03-17 13:30)
> 
> 
> Thanks,
> 
> Daniel Gergely
> 
>

Re: Review Request 44916: StackAdvisor needs to set config map for ams-grafana-env to meet non-null expectations

2016-03-18 Thread Jonathan Hurley


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44916/#review123930
---




ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/StackAdvisorBlueprintProcessor.java
 (line 224)


How can this ever be null? An empty config type!?


- Jonathan Hurley


On March 16, 2016, 4:55 p.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44916/
> ---
> 
> (Updated March 16, 2016, 4:55 p.m.)
> 
> 
> Review request for Ambari, Dmytro Sen, Jonathan Hurley, Nate Cole, and Sid 
> Wagle.
> 
> 
> Bugs: AMBARI-15442
> https://issues.apache.org/jira/browse/AMBARI-15442
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> The StackAdvisor needs to set a config map for the `ams-grafana-env`, even it 
> is empty, to meet non-null expectations in Kerberos and Blueprint logic. 
> 
> To to this, the following line needs to be added to the StackAdvisor code:
> 
> #stacks/HDP/2.0.6/services/stack_advisor.py:473
> ```
> putGrafanaProperty = self.putProperty(configurations, "ams-grafana-env", 
> services)
> ```
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/StackAdvisorBlueprintProcessor.java
>  ea9fffb 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelperImpl.java
>  be6edc9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
>  f5e7578 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
> 8c18421 
>   
> ambari-server/src/test/java/org/apache/ambari/server/api/services/stackadvisor/StackAdvisorBlueprintProcessorTest.java
>  c4e694b 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/KerberosHelperTest.java
>  f6027f3 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
>  68d5755 
>   ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py 
> 9070c0a 
> 
> Diff: https://reviews.apache.org/r/44916/diff/
> 
> 
> Testing
> ---
> 
> Manually tested in cluster Kerbrerized using the Ambari UI.
> 
> 
> Thanks,
> 
> Robert Levas
> 
>

Review Request 44962: file_system get_mount_point_for_dir works incorrect

2016-03-18 Thread Andrew Onischuk


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44962/
---

Review request for Ambari and Dmytro Sen.


Bugs: AMBARI-15468
https://issues.apache.org/jira/browse/AMBARI-15468


Repository: ambari


Description
---

**Case 1**:


[root@c6403 ~]# mount
/dev/mapper/VolGroup-lv_root on / type ext4 (rw)
/newFs on /newHdfsDir type ext3 (rw,loop=/dev/loop1)

[root@c6403 ~]# python
>>> from resource_management import *
>>> from resource_management.libraries.functions import file_system
>>> with Environment() as env:
...   print file_system.get_mount_point_for_dir('/newHdfsDir/some_dir')
...   print file_system.get_mount_point_for_dir('/newHdfsDir')
... 
2016-03-17 13:00:33,640 - Mount point for directory /newHdfsDir/some_dir is 
/
2016-03-17 13:00:33,640 - Mount point for directory /newHdfsDir is /


It should be /newHdfsDir, not /

**Reason**   
/newHdfsDir and / have the same count of os separators.

**Case 2**:


[root@c6403 ~]# mount
/dev/mapper/VolGroup-lv_root on / type ext4 (rw)
/newFs on /a/a type ext3 (rw,loop=/dev/loop2)
/newFs on /a/a1 type ext3 (rw,loop=/dev/loop3)

[root@c6403 ~]# python
>>> from resource_management import *
>>> from resource_management.libraries.functions import file_system
>>> with Environment() as env:
>>> from resource_management import *
>>> from resource_management.libraries.functions import file_system
>>> with Environment() as env:
...   print file_system.get_mount_point_for_dir('/a/a1/some_dir')
...   print file_system.get_mount_point_for_dir('/a/a1')
...
2016-03-17 13:07:38,899 - Mount point for directory /a/a1/some_dir is /a/a
2016-03-17 13:07:38,900 - Mount point for directory /a/a1 is /a/a


It should be /a/a1, not /a/a


Diffs
-

  ambari-agent/src/test/python/resource_management/TestFileSystem.py 4e0eb63 
  
ambari-common/src/main/python/resource_management/libraries/functions/file_system.py
 39b86dd 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
6132c2c 

Diff: https://reviews.apache.org/r/44962/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Re: Review Request 44933: Apply the stack featurization prototype detailed on AMBARI-13364 to ZOOKEEPER service.

2016-03-18 Thread Alejandro Fernandez


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44933/#review124298
---




ambari-server/src/main/resources/stacks/HDP/2.2/configuration/cluster-env.xml 
(line 24)


Actually, this file must exist for the entire stack and not inside the #.# 
folder.
The reason being that an EU from HDP 2.1 to 2.3 does the following:

1. Current stack is 2.1
2. Stop all services
3. Change stack to 2.3 and save new configs
4. Start all services on 2.3

Any action before step 3 may need to reference the stack_features 
min_version and max_version of the newer stack it is going to. But that will of 
course be inaccessible at that time.

So we need a single file for the entire stack name.


- Alejandro Fernandez


On March 16, 2016, 9:24 p.m., Juanjo  Marron wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44933/
> ---
> 
> (Updated March 16, 2016, 9:24 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez and Jayush Luniya.
> 
> 
> Bugs: AMBARI-14435
> https://issues.apache.org/jira/browse/AMBARI-14435
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Apply the  stack featurization prototype  detailed on AMBARI-13364 to 
> ZOOKEEPER service.
> This patch removes the hardcoded stack versions in ZK common-services code 
> (centralized version and parameterized stack configurations will be faced in 
> other JIRAS)
> stack_features is now a property in .json format in 
> configuration/cluster-env.xml for each stack version.
> With the structure:
> {
> "stack_features": [
> { "name": "feature1", "description" : "Feature1 support", "min_version" : 
> "Y.Y.Y.Y" , "max_version" : "X.X.X.X"}
> , 
> ...
> ]
> }
> where min_version/max_version are optional constraints.
> New resource_management/libraries/functions/stack_fetaures.py has been 
> introduced to parse the json file and called from service code to check if 
> the stack supports the required feature.
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/stack_fetaures.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5.2.0/package/scripts/params_linux.py
>  9ff9125 
>   
> ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5.2.0/package/scripts/status_params.py
>  d18e4d7 
>   
> ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5.2.0/package/scripts/zookeeper.py
>  5712ce4 
>   
> ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5.2.0/package/scripts/zookeeper_client.py
>  25ace24 
>   
> ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5.2.0/package/scripts/zookeeper_server.py
>  d5b6898 
>   
> ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5.2.0/package/scripts/zookeeper_service.py
>  6fe0772 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/configuration/cluster-env.xml
>  3fb82e9 
>   
> ambari-server/src/main/resources/stacks/HDP/2.2/configuration/cluster-env.xml 
> f1fa4de 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/configuration/cluster-env.xml 
> PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/44933/diff/
> 
> 
> Testing
> ---
> 
> Zookeeper fresh installation has been tested using HDP stack 2.3 and 
> AMBARI-13364 branch code.
> 
> 
> Thanks,
> 
> Juanjo  Marron
> 
>

Re: Review Request 45054: AMBARI-15443:Make Host bulk command menu item list stack driven instead of a hardcoded list in UI code

2016-03-18 Thread Alejandro Fernandez


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45054/#review124303
---


Fix it, then Ship it!




Thank you for adding this. Any unit tests?


ambari-server/src/main/java/org/apache/ambari/server/controller/StackServiceComponentResponse.java
 (line 144)


Style, prefer to use braces after "if"
Add java doc


- Alejandro Fernandez


On March 18, 2016, 9:46 p.m., Di Li wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45054/
> ---
> 
> (Updated March 18, 2016, 9:46 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jaimin Jetly, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-15443
> https://issues.apache.org/jira/browse/AMBARI-15443
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> The items described here are the ones on the Hosts tab, in the Actions 
> drop-down list where the UI shows entries such as All Hosts. If user mouses 
> over the All Hosts, it shows a sub-list including Hosts and slave components.
> 
> The slave component items are hardcoded in hosts_table_menu_view.js as shown 
> below. This jira is to put this info into each service's metainfo.xml so that 
> the slave component items can be stack driven.
> 
> components: function () {
> var serviceNames = App.Service.find().mapProperty('serviceName');
> var menuItems = [
> O.create(
> { serviceName: 'HDFS', componentName: 'DATANODE', masterComponentName: 
> 'NAMENODE', componentNameFormatted: 
> Em.I18n.t('dashboard.services.hdfs.datanodes') }
> 
> ),
> O.create(
> { serviceName: 'YARN', componentName: 'NODEMANAGER', masterComponentName: 
> 'RESOURCEMANAGER', componentNameFormatted: 
> Em.I18n.t('dashboard.services.yarn.nodeManagers') }
> 
> ),
> O.create(
> { serviceName: 'HBASE', componentName: 'HBASE_REGIONSERVER', 
> masterComponentName: 'HBASE_MASTER', componentNameFormatted: 
> Em.I18n.t('dashboard.services.hbase.regionServers') }
> 
> ),
> O.create(
> { serviceName: 'STORM', componentName: 'SUPERVISOR', masterComponentName: 
> 'SUPERVISOR', componentNameFormatted: 
> Em.I18n.t('dashboard.services.storm.supervisors') }
> 
> )];
> 
> return menuItems.filter(function (item)
> { return serviceNames.contains(item.serviceName); }
> 
> );
> }.property(),
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/StackServiceComponentResponse.java
>  cfd4e7b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackServiceComponentResourceProvider.java
>  70945ba 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ComponentModule.java
>  a122dc6 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/BulkCommandDefinition.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/ComponentInfo.java 
> 770ee5c 
>   ambari-server/src/main/resources/common-services/HAWQ/2.0.0/metainfo.xml 
> e35b7d8 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/metainfo.xml
>  057e126 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml 
> 75d3bea 
>   ambari-server/src/main/resources/common-services/PXF/3.0.0/metainfo.xml 
> afe27ec 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/metainfo.xml 
> 804374a 
>   
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/metainfo.xml 
> 0f71585 
>   ambari-server/src/main/resources/properties.json 01c15f2 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/ComponentModuleTest.java
>  2f84f04 
>   ambari-web/app/mappers/stack_service_mapper.js 8a65055 
>   ambari-web/app/models/stack_service_component.js 26ff1b8 
>   ambari-web/app/views/main/host/hosts_table_menu_view.js e75b643 
>   ambari-web/test/mappers/stack_service_mapper_test.js 4bc36fe 
> 
> Diff: https://reviews.apache.org/r/45054/diff/
> 
> 
> Testing
> ---
> 
> unit test
> patch a trunk cluster with code change, verify the host bulk command list 
> shown for DataNode, HBase Region server, storm supervisor, etc.
> 
> 
> Thanks,
> 
> Di Li
> 
>

Re: Review Request 45069: HiveServerInteractive. Remove Password requirement from hive-interactive-site.xml.

2016-03-18 Thread Yusaku Sako


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45069/#review124354
---


Ship it!




Ship It!

- Yusaku Sako


On March 19, 2016, 1:43 a.m., Swapan Shridhar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45069/
> ---
> 
> (Updated March 19, 2016, 1:43 a.m.)
> 
> 
> Review request for Ambari, Jaimin Jetly, Sumit Mohanty, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-15486
> https://issues.apache.org/jira/browse/AMBARI-15486
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> - HiveServerInteractive. Remove Password requirement from 
> hive-interactive-site.xml.
> - Reason : QE tests are failing as pawwors is being asked twice.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/HDP/2.6/services/HIVE/configuration/hive-interactive-site.xml
>  e77b379 
> 
> Diff: https://reviews.apache.org/r/45069/diff/
> 
> 
> Testing
> ---
> 
> Yes. Password not asked twice.
> 
> 
> Thanks,
> 
> Swapan Shridhar
> 
>

Re: Review Request 44986: AMBARI-15474: Listen for changes to auto-start configuration and send them to the agent during heartbeats

2016-03-18 Thread Nahappan Somasundaram

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44986/
---

(Updated March 18, 2016, 11:07 a.m.)

Review request for Ambari, Jonathan Hurley, Nate Cole, Sumit Mohanty, and Sid
Wagle.

Changes
---

1. Fixed Log.info() to use {}
2. Renamed ServiceComponentRecoveryChangeEvent to
ServiceComponentRecoveryChangedEvent

Bugs: AMBARI-15474
https://issues.apache.org/jira/browse/AMBARI-15474

Repository: ambari

Description
---

AMBARI-15474: Listen for changes to auto-start configuration and send them to
the agent during heartbeats.

In HeartbeatHandler::handleHeartBeat(), send the recovery configuration to the
agent. Instead of pulling the recovery configuration from the application
cache, implement change events and get notified whenever there are changes to
the recovery configuration.

**Changes**:
1. Instead of sending the recovery configuration to the agent during every
heartbeat (10 seconds interval), send the configuration only when changes are
detected.
2. Used AmbariEventPublisher to publish the changes. RecoveryConfigHelper is
the subscriber which listens to changes to the configuration.
3. RecoveryConfigHelper maintains a map of Cluster-->Host-->Timestamp which is
the last updated timestamp of the recovery configuration. This timestamp is
used as a token and is sent to the agent with the configuration. The agent
provides this timestamp during each heartbeat. If the timestamp from the
heartbeat is the same as the one on the server side, no configuration is sent
to the agent.
4. Whenever changes are detected, the timestamp is invalidated. During the next
heartbeat, the recovery configuration is read from the DB and sent to the agent
along with the latest timestamp.
5. Added new unit tests to test each configuration change event.

Diffs (updated)
-

ambari-agent/src/main/python/ambari_agent/Controller.py
c1c16ac97cca0d2677da2e26eee7b4949a4bf15c
ambari-agent/src/main/python/ambari_agent/Heartbeat.py
b28644c834a2387d2fb0ad17d104224e830a0245
ambari-agent/src/main/python/ambari_agent/RecoveryManager.py
ed537ca6928ce376c5f3e906f7a47c3f1919e309
ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeat.java
1430aa20cf68db071be85c4ad2ebb9acdaccecc0

ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java
3a80803724bcb6ca9e4ec875aa17c9ac1c66fbe8

ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatResponse.java
617b04b5b8768ba28a3c8ecb6e5e00601f153396

ambari-server/src/main/java/org/apache/ambari/server/agent/RecoveryConfig.java
c2c18462c586292caaf40bdb6a25a8fe9a39d76c

ambari-server/src/main/java/org/apache/ambari/server/agent/RecoveryConfigHelper.java
92a622117c71ddfc3bd2562c04ee525491bd6ffd
ambari-server/src/main/java/org/apache/ambari/server/events/AmbariEvent.java
78731854fe80c3d6087a73ebf00b0ffe9e287354

ambari-server/src/main/java/org/apache/ambari/server/events/ClusterConfigChangedEvent.java
PRE-CREATION

ambari-server/src/main/java/org/apache/ambari/server/events/ServiceComponentInstalledEvent.java
2ce01236b3bb1ab4934fe456e5669fa474987426

ambari-server/src/main/java/org/apache/ambari/server/events/ServiceComponentRecoveryChangedEvent.java
PRE-CREATION

ambari-server/src/main/java/org/apache/ambari/server/events/ServiceComponentUninstalledEvent.java
19712cdb18592816f82898cd7aa08a3600c7e1f4
ambari-server/src/main/java/org/apache/ambari/server/state/ConfigImpl.java
2cc3d0013140c9c7d3453d7c5e05c3aa02bc5794

ambari-server/src/main/java/org/apache/ambari/server/state/ServiceComponentImpl.java
4be1c21bf41416681be836716ce38305cc3f287e

ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java
3a0075e3a3c90088ca6bd2d9eca48ef50db9fdb3

ambari-server/src/test/java/org/apache/ambari/server/agent/TestHeartbeatHandler.java
b1f94e38f97550123dd366984438b0bbf9c2826c

ambari-server/src/test/java/org/apache/ambari/server/configuration/RecoveryConfigHelperTest.java
c86b95a7cd10b0720f0a3b9676f65a8bd68dd6c9

ambari-server/src/test/java/org/apache/ambari/server/events/listeners/upgrade/HostVersionOutOfSyncListenerTest.java
1bf9d8336533bc677384314f998f10aa2f10504e

Diff: https://reviews.apache.org/r/44986/diff/

Testing
---

**1. mvn clean install **

[INFO]
[INFO] Reactor Summary:
[INFO]
[INFO] Ambari Main ... SUCCESS [5.864s]
[INFO] Apache Ambari Project POM . SUCCESS [0.037s]
[INFO] Ambari Web SUCCESS [23.683s]
[INFO] Ambari Views .. SUCCESS [1.069s]
[INFO] Ambari Admin View . SUCCESS [5.949s]
[INFO] ambari-metrics

Re: Review Request 45012: Recommendation and validation logic for HAWQ hdfs-client.xml output.replace-datanode-on-failure property

2016-03-18 Thread bhuvnesh chaudhary


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45012/#review124226
---


Ship it!




Ship It!

- bhuvnesh chaudhary


On March 18, 2016, 2:01 a.m., Alexander Denissov wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45012/
> ---
> 
> (Updated March 18, 2016, 2:01 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, bhuvnesh chaudhary, Goutam 
> Tadi, jun aoki, Jayush Luniya, Lav Jain, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-15499
> https://issues.apache.org/jira/browse/AMBARI-15499
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Recommendation and validation logic for HAWQ hdfs-client.xml 
> output.replace-datanode-on-failure property
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hdfs-client.xml
>  c2a756a 
>   ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 
> c0bc4e2 
>   ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 
> 4f0b4b7 
> 
> Diff: https://reviews.apache.org/r/45012/diff/
> 
> 
> Testing
> ---
> 
> manual
> 
> --
> Ran 253 tests in 7.111s
> 
> OK
> --
> Total run:927
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Alexander Denissov
> 
>

Re: Review Request 44987: Editing tmp dir from single to multiple dir doesn't trigger the backend change

2016-03-18 Thread jun aoki


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44987/
---

(Updated March 18, 2016, 10:17 p.m.)


Review request for Ambari.


Bugs: AMBARI-15475
https://issues.apache.org/jira/browse/AMBARI-15475


Repository: ambari


Description
---

Modified "HAWQ Segment Temp Directory” from one [/d1/tmp/hawqsegment] to many 
[/d1/tmp/hawqsegment,/d2/tmp/hawqsegment,/d3/tmp/hawqsegment,/d4/tmp/hawqsegment,/d5/tmp/hawqsegment]
and the change is not propagated to segments and the folders are not created.


Diffs (updated)
-

  
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/common.py
 d62332d 
  
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/hawqsegment.py
 0a597b6 
  
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/master_helper.py
 9482b35 
  
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/params.py
 1bdc5aa 
  
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/utils.py
 dc6c203 
  ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqmaster.py 14b5a5f 
  ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqsegment.py 6ca83b7 
  ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqstandby.py eaba496 

Diff: https://reviews.apache.org/r/44987/diff/


Testing
---

manually tested.

test_configure_default (test_hawqsegment.TestHawqSegment) ... ok
test_install_default (test_hawqsegment.TestHawqSegment) ... ok
test_start_default (test_hawqsegment.TestHawqSegment) ... ok
test_stop_default (test_hawqsegment.TestHawqSegment) ... ok
test_hawq_master_critical 
(test_alert_component_status.TestAlertComponentStatus) ... ok
test_hawq_master_ok (test_alert_component_status.TestAlertComponentStatus) ... 
ok
test_hawq_segment_critical 
(test_alert_component_status.TestAlertComponentStatus) ... ok
test_hawq_segment_ok (test_alert_component_status.TestAlertComponentStatus) ... 
ok
test_hawq_standby_critical 
(test_alert_component_status.TestAlertComponentStatus) ... ok
test_hawq_standby_ok (test_alert_component_status.TestAlertComponentStatus) ... 
ok
test_missing_configs (test_alert_component_status.TestAlertComponentStatus) ... 
ok
test_configure_default (test_hawqstandby.TestHawqStandby) ... ok
test_install_default (test_hawqstandby.TestHawqStandby) ... ok
test_start_default (test_hawqstandby.TestHawqStandby) ... 2016-03-17 
15:33:42,969 - Standby host c6401.ambari.apache.org initialized
ok
test_stop_default (test_hawqstandby.TestHawqStandby) ... ok
test_configure_default (test_hawqmaster.TestHawqMaster) ... ok
test_install_default (test_hawqmaster.TestHawqMaster) ... ok
test_start_default (test_hawqmaster.TestHawqMaster) ... 2016-03-17 15:33:43,025 
- Active master c6401.ambari.apache.org initialized
ok
test_start_localmaster (test_hawqmaster.TestHawqMaster) ... 2016-03-17 
15:33:43,049 - Master c6401.ambari.apache.org started
ok
test_stop_default (test_hawqmaster.TestHawqMaster) ... ok


Thanks,

jun aoki

Re: Review Request 44972: Improve error logging for install errors during blueprint deployments.

2016-03-18 Thread Di Li


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44972/#review124176
---


Ship it!




Ship It!

- Di Li


On March 17, 2016, 6:11 p.m., Amruta Borkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44972/
> ---
> 
> (Updated March 17, 2016, 6:11 p.m.)
> 
> 
> Review request for Ambari, Di Li and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-15412
> https://issues.apache.org/jira/browse/AMBARI-15412
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Improve error logging for install errors during blueprint deployments. 
> 
> Currently a severe error during install of a service component gets logged as 
> a WARNing
> 
> E.g.:
> 
> 09 Mar 2016 12:11:45,881 WARN [qtp-ambari-agent-146] HeartBeatHandler:603 - 
> Operation failed - may be retried. Service component host: KAFKA_BROKER, 
> host: hdtest159.svl.ibm.com Action id12-0
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java
>  24fea22 
> 
> Diff: https://reviews.apache.org/r/44972/diff/
> 
> 
> Testing
> ---
> 
> There are no JUnit test cases, but attached the screenshot which shows 
> modified output.
> 
> 
> File Attachments
> 
> 
> Patch
>   
> https://reviews.apache.org/media/uploaded/files/2016/03/17/2110d517-0cf4-4477-87e7-935c963e04de__AMBARI-15412.patch
> Output
>   
> https://reviews.apache.org/media/uploaded/files/2016/03/17/923d218f-7ca4-4439-b42f-743511936f94__AMBARI-15412_output.png
> 
> 
> Thanks,
> 
> Amruta Borkar
> 
>

Re: Review Request 45030: Replace 0.0.0.0 with hostname in ams-site/timeline.metrics.service.webapp.address

Re: Review Request 45035: Restarting HDFS Before Upgrade Finalizing Does Not Supply the rollingUpgrade Flag

Re: Review Request 44986: AMBARI-15474: Listen for changes to auto-start configuration and send them to the agent during heartbeats

Re: Review Request 44983: Service version display should be based on Version Definition

Review Request 44987: Editing tmp dir from single to multiple dir doesn't trigger the backend change

Re: Review Request 45054: AMBARI-15443:Make Host bulk command menu item list stack driven instead of a hardcoded list in UI code

Re: Review Request 45030: Replace 0.0.0.0 with hostname in ams-site/timeline.metrics.service.webapp.address

Re: Review Request 44931: Use Version Definition value for package-version when installing

Re: Review Request 44931: Use Version Definition value for package-version when installing

Re: Review Request 44956: Topology host info is not cleared when a host is removed

Re: Review Request 44886: AMBARI-15437 : Handle changing symlinks of ranger admin binaries for upgrade scenarios

Re: Review Request 44835: AMBARI-15420: Refactor resource_management library

Review Request 45056: Blueprint install using config_recommendation_strategy is not functional

Re: Review Request 44933: Apply the stack featurization prototype detailed on AMBARI-13364 to ZOOKEEPER service.

Re: Review Request 44958: Increase Ambari Server Perm gen default value

Re: Review Request 44969: After upgrading from Ambari 2.1.2 to 2.2.1, HDP from 2.3.0 to 2.3.4 , user is getting 'Consistency Check Failed' error while trying to save configuration in Ranger, Hive and

Re: Review Request 44956: Topology host info is not cleared when a host is removed

Re: Review Request 44916: StackAdvisor needs to set config map for ams-grafana-env to meet non-null expectations

Review Request 44962: file_system get_mount_point_for_dir works incorrect

Re: Review Request 44933: Apply the stack featurization prototype detailed on AMBARI-13364 to ZOOKEEPER service.

Re: Review Request 45054: AMBARI-15443:Make Host bulk command menu item list stack driven instead of a hardcoded list in UI code

Re: Review Request 45069: HiveServerInteractive. Remove Password requirement from hive-interactive-site.xml.

Re: Review Request 44986: AMBARI-15474: Listen for changes to auto-start configuration and send them to the agent during heartbeats

Re: Review Request 45012: Recommendation and validation logic for HAWQ hdfs-client.xml output.replace-datanode-on-failure property

Re: Review Request 44987: Editing tmp dir from single to multiple dir doesn't trigger the backend change

Re: Review Request 44972: Improve error logging for install errors during blueprint deployments.

26 matches

Site Navigation

Mail list logo

Footer information