date:20160319


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44956/#review124068
---


Fix it, then Ship it!





ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClustersImpl.java
 (line 910)


Move up a couple of lines to be with the other DAOs.



ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog222.java
 (line 145)


May need to ensure this is idempotent.


- Alejandro Fernandez


On March 17, 2016, 12:59 p.m., Daniel Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44956/
> ---
> 
> (Updated March 17, 2016, 12:59 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Laszlo Puskas, Oliver Szabo, 
> Sandor Magyari, Sumit Mohanty, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-15457
> https://issues.apache.org/jira/browse/AMBARI-15457
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When a host is removed, a record is left in the topology_host_info table. As 
> a result re-adding the same host is not possible, due to invalid topology.
> Host removal should remove the corresponding record from topology_host_info 
> as well.
> 
> A new column with a foreign key to hosts table is added. In addition 
> ClusterTopologyImpl was statefull and hostGroupInfoMap was not updated when a 
> host was removed.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostResourceProvider.java
>  6101ee8 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/TopologyHostInfoDAO.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/TopologyHostInfoEntity.java
>  a4f251a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClustersImpl.java
>  46dd49d 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterTopology.java
>  4e178c0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterTopologyImpl.java
>  003539c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/HostGroupInfo.java
>  4847a11 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/PersistedState.java
>  dbf6735 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/PersistedStateImpl.java
>  b878955 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/TopologyManager.java
>  41f3150 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog222.java
>  8267d5d 
>   ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql dee397f 
>   ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 253b5d7 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 832d1bc 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql 
> 6c8a09d 
>   ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql a6acb74 
>   ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 2957c5c 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerTest.java
>  3e43ef1 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/HostResourceProviderTest.java
>  69c570e 
> 
> Diff: https://reviews.apache.org/r/44956/diff/
> 
> 
> Testing
> ---
> 
> Tests ran without failures. (2016-03-17 13:30)
> 
> 
> Thanks,
> 
> Daniel Gergely
> 
>

Re: Review Request 44962: file_system get_mount_point_for_dir works incorrect

2016-03-19 Thread Dmytro Sen


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44962/#review124045
---


Ship it!




Ship It!

- Dmytro Sen


On Март 17, 2016, 4:03 п.п., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44962/
> ---
> 
> (Updated Март 17, 2016, 4:03 п.п.)
> 
> 
> Review request for Ambari, Alejandro Fernandez and Dmytro Sen.
> 
> 
> Bugs: AMBARI-15468
> https://issues.apache.org/jira/browse/AMBARI-15468
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> **Case 1**:
> 
> 
> [root@c6403 ~]# mount
> /dev/mapper/VolGroup-lv_root on / type ext4 (rw)
> /newFs on /newHdfsDir type ext3 (rw,loop=/dev/loop1)
> 
> [root@c6403 ~]# python
> >>> from resource_management import *
> >>> from resource_management.libraries.functions import file_system
> >>> with Environment() as env:
> ...   print file_system.get_mount_point_for_dir('/newHdfsDir/some_dir')
> ...   print file_system.get_mount_point_for_dir('/newHdfsDir')
> ... 
> 2016-03-17 13:00:33,640 - Mount point for directory /newHdfsDir/some_dir 
> is /
> 2016-03-17 13:00:33,640 - Mount point for directory /newHdfsDir is /
> 
> 
> It should be /newHdfsDir, not /
> 
> **Reason**   
> /newHdfsDir and / have the same count of os separators.
> 
> **Case 2**:
> 
> 
> [root@c6403 ~]# mount
> /dev/mapper/VolGroup-lv_root on / type ext4 (rw)
> /newFs on /a/a type ext3 (rw,loop=/dev/loop2)
> /newFs on /a/a1 type ext3 (rw,loop=/dev/loop3)
> 
> [root@c6403 ~]# python
> >>> from resource_management import *
> >>> from resource_management.libraries.functions import file_system
> >>> with Environment() as env:
> >>> from resource_management import *
> >>> from resource_management.libraries.functions import file_system
> >>> with Environment() as env:
> ...   print file_system.get_mount_point_for_dir('/a/a1/some_dir')
> ...   print file_system.get_mount_point_for_dir('/a/a1')
> ...
> 2016-03-17 13:07:38,899 - Mount point for directory /a/a1/some_dir is /a/a
> 2016-03-17 13:07:38,900 - Mount point for directory /a/a1 is /a/a
> 
> 
> It should be /a/a1, not /a/a
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/test/python/resource_management/TestFileSystem.py 4e0eb63 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/file_system.py
>  39b86dd 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
> 6132c2c 
> 
> Diff: https://reviews.apache.org/r/44962/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 44923: Alerts: Allow alerts to emit a structured response


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44923/
---

(Updated Березень 17, 2016, 8:11 після полудня)


Review request for Ambari, Dmytro Sen and Jonathan Hurley.


Bugs: AMBARI-15445
https://issues.apache.org/jira/browse/AMBARI-15445


Repository: ambari


Description
---

For some alerts, having the ability to include a link in the response would be 
incredibly beneficial.  Having a structured response format would allow alert 
creators to add links that operators could click on to help give more context 
around an alert, or point to a remediation step.  Something like:

{code}
{
  "response": "There were more than 5 soft limits reached in the past 10 
minutes.  Please click the link to see the log file.",
  "href": "http://some.server/other/stuff;
}
{code}


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AlertDefinitionResourceProvider.java
 0f73ec6 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/AlertDefinitionEntity.java
 9e21bec 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/AlertDefinitionEntity_.java
 d2d4cf3 
  
ambari-server/src/main/java/org/apache/ambari/server/state/alert/AlertDefinition.java
 9fff5f2 
  
ambari-server/src/main/java/org/apache/ambari/server/state/alert/AlertDefinitionFactory.java
 3b4f5fc 
  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
 a803f73 
  ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql a85202d 
  ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 9b4810c 
  ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql cc3d197 
  ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 07c786d 
  ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql 
ab6dc93 
  ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 8e91fde 
  ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 440ca44 

Diff: https://reviews.apache.org/r/44923/diff/


Testing
---

Tests will be added after patch draft review


Thanks,

Vitalyi Brodetskyi

Re: Review Request 44880: AMBARI-15436 Supportability: Create 'Reset UI' button in Ambari's experimental page

2016-03-19 Thread Zhe (Joe) Wang


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44880/
---

(Updated March 17, 2016, 8:52 p.m.)


Review request for Ambari, Jaimin Jetly, Richard Zang, Srimanth Gunturi, Xi 
Wang, and Yusaku Sako.


Bugs: AMBARI-15436
https://issues.apache.org/jira/browse/AMBARI-15436


Repository: ambari


Description
---

Currently Ambari UI persists it's current state on the server. This helps when 
a different user logs in, they end up in the same UI. Sometimes however, UI's 
state persisted on server does not sync with cluster state, thereby putting UI 
in a broken state with no recourse for the stuck user.

Ambari UI should provide a _Reset UI_ button in the experimental page which 
resets the UI state on server.


Diffs (updated)
-

  ambari-web/app/controllers/experimental.js 9fa6da5 
  ambari-web/app/mappers/server_data_mapper.js 1e08161 
  ambari-web/app/messages.js 9289fc2 
  ambari-web/app/templates/experimental.hbs 2873679 

Diff: https://reviews.apache.org/r/44880/diff/


Testing
---

Local ambari-web test passed.
24578 tests complete (23 seconds)
145 tests pending
Manual testing done.


Thanks,

Zhe (Joe) Wang

Re: Review Request 44959: Enable Ambari build process to automatically switch repo base urls in the repo files

2016-03-19 Thread Sumit Mohanty


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44959/#review124020
---


Ship it!




Ship It!

- Sumit Mohanty


On March 17, 2016, 2:49 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44959/
> ---
> 
> (Updated March 17, 2016, 2:49 p.m.)
> 
> 
> Review request for Ambari and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-15467
> https://issues.apache.org/jira/browse/AMBARI-15467
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Today, there is a manual step, at the end of each release to update the repo
> baseurls from internal dev repo to external public repo. This is error prone
> and easy to miss.
> 
> Ambari build process should be modified to allow RE to specify the public repo
> urls as the baseUrl for the repos.
> 
> 
> Diffs
> -
> 
>   ambari-common/src/main/python/urlinfo_processor/urlinfo_processor.py 
> PRE-CREATION 
>   ambari-server/pom.xml 5bb3b98 
> 
> Diff: https://reviews.apache.org/r/44959/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Review Request 44933: Apply the stack featurization prototype detailed on AMBARI-13364 to ZOOKEEPER service.

2016-03-19 Thread Juanjo Marron


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44933/
---

Review request for Ambari, Alejandro Fernandez and Jayush Luniya.


Bugs: AMBARI-14435
https://issues.apache.org/jira/browse/AMBARI-14435


Repository: ambari


Description
---

Apply the  stack featurization prototype  detailed on AMBARI-13364 to ZOOKEEPER 
service.
This patch removes the hardcoded stack versions in ZK common-services code 
(centralized version and parameterized stack configurations will be faced in 
other JIRAS)
stack_features is now a property in .json format in 
configuration/cluster-env.xml for each stack version.
With the structure:
{
"stack_features": [
{ "name": "feature1", "description" : "Feature1 support", "min_version" : 
"Y.Y.Y.Y" , "max_version" : "X.X.X.X"}
, 
...
]
}
where min_version/max_version are optional constraints.
New resource_management/libraries/functions/stack_fetaures.py has been 
introduced to parse the json file and called from service code to check if the 
stack supports the required feature.


Diffs
-

  
ambari-common/src/main/python/resource_management/libraries/functions/stack_fetaures.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5.2.0/package/scripts/params_linux.py
 9ff9125 
  
ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5.2.0/package/scripts/status_params.py
 d18e4d7 
  
ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5.2.0/package/scripts/zookeeper.py
 5712ce4 
  
ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5.2.0/package/scripts/zookeeper_client.py
 25ace24 
  
ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5.2.0/package/scripts/zookeeper_server.py
 d5b6898 
  
ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5.2.0/package/scripts/zookeeper_service.py
 6fe0772 
  
ambari-server/src/main/resources/stacks/HDP/2.0.6/configuration/cluster-env.xml 
3fb82e9 
  ambari-server/src/main/resources/stacks/HDP/2.2/configuration/cluster-env.xml 
f1fa4de 
  ambari-server/src/main/resources/stacks/HDP/2.3/configuration/cluster-env.xml 
PRE-CREATION 

Diff: https://reviews.apache.org/r/44933/diff/


Testing
---

Zookeeper fresh installation has been tested using HDP stack 2.3 and 
AMBARI-13364 branch code.


Thanks,

Juanjo  Marron

Review Request 44956: Topology host info is not cleared when a host is removed

2016-03-19 Thread Daniel Gergely


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44956/
---

Review request for Ambari, Alejandro Fernandez, Laszlo Puskas, Oliver Szabo, 
Sandor Magyari, Sumit Mohanty, and Sebastian Toader.


Bugs: AMBARI-15457
https://issues.apache.org/jira/browse/AMBARI-15457


Repository: ambari


Description
---

When a host is removed, a record is left in the topology_host_info table. As a 
result re-adding the same host is not possible, due to invalid topology.
Host removal should remove the corresponding record from topology_host_info as 
well.

A new column with a foreign key to hosts table is added. In addition 
ClusterTopologyImpl was statefull and hostGroupInfoMap was not updated when a 
host was removed.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostResourceProvider.java
 6101ee8 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/dao/TopologyHostInfoDAO.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/TopologyHostInfoEntity.java
 a4f251a 
  
ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClustersImpl.java
 46dd49d 
  
ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterTopology.java
 4e178c0 
  
ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterTopologyImpl.java
 003539c 
  
ambari-server/src/main/java/org/apache/ambari/server/topology/HostGroupInfo.java
 4847a11 
  
ambari-server/src/main/java/org/apache/ambari/server/topology/PersistedState.java
 dbf6735 
  
ambari-server/src/main/java/org/apache/ambari/server/topology/PersistedStateImpl.java
 b878955 
  
ambari-server/src/main/java/org/apache/ambari/server/topology/TopologyManager.java
 41f3150 
  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog222.java
 8267d5d 
  ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql dee397f 
  ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 253b5d7 
  ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 832d1bc 
  ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql 
6c8a09d 
  ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql a6acb74 
  ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 2957c5c 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerTest.java
 3e43ef1 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/HostResourceProviderTest.java
 69c570e 

Diff: https://reviews.apache.org/r/44956/diff/


Testing
---

Tests ran without failures. (2016-03-17 13:30)


Thanks,

Daniel Gergely

Re: Review Request 45012: Recommendation and validation logic for HAWQ hdfs-client.xml output.replace-datanode-on-failure property


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45012/
---

(Updated March 18, 2016, 7:34 p.m.)


Review request for Ambari, Alejandro Fernandez, bhuvnesh chaudhary, Goutam 
Tadi, jun aoki, Jayush Luniya, Lav Jain, and Sumit Mohanty.


Changes
---

changed to lower-case booleans, updated some descriptions


Bugs: AMBARI-15499
https://issues.apache.org/jira/browse/AMBARI-15499


Repository: ambari


Description
---

Recommendation and validation logic for HAWQ hdfs-client.xml 
output.replace-datanode-on-failure property


Diffs (updated)
-

  
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-site.xml
 93ad3fe 
  
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hdfs-client.xml
 c2a756a 
  ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 
c0bc4e2 
  ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 4f0b4b7 

Diff: https://reviews.apache.org/r/45012/diff/


Testing
---

manual

--
Ran 253 tests in 7.111s

OK
--
Total run:927
Total errors:0
Total failures:0
OK


Thanks,

Alexander Denissov

Re: Review Request 44886: AMBARI-15437 : Handle changing symlinks of ranger admin binaries for upgrade scenarios

2016-03-19 Thread Gautam Borad



> On March 16, 2016, 12:24 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py,
> >  lines 128-131
> > 
> >
> > Do you need to:
> > 
> > - Check for the existance of these links first before removing/creating 
> > them?
> > - Check a specific version of the stack that you are upgrading from/to?

1) No, if it already exists, we delete it. The Link() does not throw error, if 
it does not exists in the first place.
2) /usr/bin/ranger-admin is used from stack 2.3 onwards (Also, please note from 
2.3 stack onwards, Ranger uses setup_ranger_xml.py)


- Gautam


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44886/#review123860
---


On March 16, 2016, 9:15 a.m., Gautam Borad wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44886/
> ---
> 
> (Updated March 16, 2016, 9:15 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jaimin Jetly, Jonathan 
> Hurley, Mahadev Konar, Sumit Mohanty, Selvamohan Neethiraj, and Velmurugan 
> Periasamy.
> 
> 
> Bugs: AMBARI-15437
> https://issues.apache.org/jira/browse/AMBARI-15437
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> For Ambari stacks for Ranger: 
> We need to take care of handling symlinks to point to current version of HDP, 
> to avoid confusion.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py
>  6394f51 
> 
> Diff: https://reviews.apache.org/r/44886/diff/
> 
> 
> Testing
> ---
> 
> Tested EU and RU with Ranger installed.
> Testing steps:
> 1) Installed Ranger with HDP-2.2.x
> 2) Upgrade Ranger to HDP-2.3.x
> 3) Again Upgrade to HDP-2.4.x
> 
> Ran 277 tests in 7.293s
> 
> OK
> --
> Total run:941
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Gautam Borad
> 
>

Re: Review Request 44958: Increase Ambari Server Perm gen default value


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44958/#review124172
---


Ship it!




Ship It!

- Robert Levas


On March 18, 2016, 4:39 a.m., Daniel Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44958/
> ---
> 
> (Updated March 18, 2016, 4:39 a.m.)
> 
> 
> Review request for Ambari, Laszlo Puskas, Oliver Szabo, Robert Levas, Sandor 
> Magyari, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-15465
> https://issues.apache.org/jira/browse/AMBARI-15465
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Increased permGen size to 128m, also added **-XX:+CMSPermGenSweepingEnabled** 
> and **-XX:+CMSClassUnloadingEnabled** to allow jvm to keep permGen space 
> clean.
> 
> 
> Diffs
> -
> 
>   ambari-server/conf/unix/ambari-env.sh e9fdcdd 
>   ambari-server/src/main/python/ambari_server_main.py 00e084b 
> 
> Diff: https://reviews.apache.org/r/44958/diff/
> 
> 
> Testing
> ---
> 
> Manual testing:
> [root@c6401 sbin]# ps -ef | grep ambari-server
> root 20769 1 27 14:47 pts/000:00:24 
> //usr/jdk64/jdk1.8.0_60/bin/java -server -XX:NewRatio=3 
> -XX:+UseConcMarkSweepGC -XX:-UseGCOverheadLimit 
> -XX:CMSInitiatingOccupancyFraction=60 **-XX:+CMSPermGenSweepingEnabled 
> -XX:+CMSClassUnloadingEnabled** -Dsun.zip.disableMemoryMapping=true -Xms512m 
> -Xmx2048m **-XX:MaxPermSize=128m** 
> -Djava.security.auth.login.config=//etc/ambari-server/conf/krb5JAASLogin.conf 
> -Djava.security.krb5.conf=/etc/krb5.conf 
> -Djavax.security.auth.useSubjectCredsOnly=false -Xms512m -Xmx2048m 
> -Djava.security.auth.login.config=//etc/ambari-server/conf/krb5JAASLogin.conf 
> -Djava.security.krb5.conf=/etc/krb5.conf 
> -Djavax.security.auth.useSubjectCredsOnly=false -cp 
> //etc/ambari-server/conf:/usr/lib/ambari-server/*:/usr/share/java/postgresql-jdbc.jar
>  org.apache.ambari.server.controller.AmbariServer
> 
> 
> Thanks,
> 
> Daniel Gergely
> 
>

Review Request 44983: Service version display should be based on Version Definition


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44983/
---

Review request for Ambari, Alejandro Fernandez and Jonathan Hurley.


Bugs: AMBARI-15473
https://issues.apache.org/jira/browse/AMBARI-15473


Repository: ambari


Description
---

Add {{stack_services}} to the repository_version endpoint, similar to the 
{{services}} attribute that lists the available services.

This will return a list of all stack services as they pertain to the Version 
Definition File.  This is required to show the correct versions that are 
available for a stack based on the repo_version.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RepositoryVersionResourceProvider.java
 c298e0a 
  
ambari-server/src/main/java/org/apache/ambari/server/state/repository/ManifestServiceInfo.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/state/repository/VersionDefinitionXml.java
 93ac767 
  
ambari-server/src/test/java/org/apache/ambari/server/state/repository/VersionDefinitionTest.java
 f2939c7 
  ambari-server/src/test/resources/version_definition_test.xml 69ea581 

Diff: https://reviews.apache.org/r/44983/diff/


Testing
---

Manual.  Automated pending.


Thanks,

Nate Cole

Re: Review Request 44773: Hive View : Upload Table : Tables are not deleted in case of exception.

2016-03-19 Thread Pallav Kulshreshtha


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44773/#review123976
---


Ship it!




Ship It!

- Pallav Kulshreshtha


On March 14, 2016, 6:16 a.m., Nitiraj Rathore wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44773/
> ---
> 
> (Updated March 14, 2016, 6:16 a.m.)
> 
> 
> Review request for Ambari, DIPAYAN BHOWMICK, Pallav Kulshreshtha, and Yusaku 
> Sako.
> 
> 
> Bugs: AMBARI-15392
> https://issues.apache.org/jira/browse/AMBARI-15392
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Earlier : 
> In case of error the temporary table and actual table is was not getting 
> cleanedup (deleted).
> 
> In this patch:
> Now the rollback methods are added to UI which sends proper REST calls to 
> server for deletion of appropriate tables depending on the stage of flow 
> where error occurs.
> 
> 
> Diffs
> -
> 
>   
> contrib/views/hive/src/main/resources/ui/hive-web/app/controllers/upload-table.js
>  5085cc5 
> 
> Diff: https://reviews.apache.org/r/44773/diff/
> 
> 
> Testing
> ---
> 
> Manual testing done.
> 
> 
> Thanks,
> 
> Nitiraj Rathore
> 
>

Re: Review Request 45035: Restarting HDFS Before Upgrade Finalizing Does Not Supply the rollingUpgrade Flag


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45035/
---

(Updated March 18, 2016, 4:56 p.m.)


Review request for Ambari, Alejandro Fernandez, Nate Cole, and Robert Levas.


Changes
---

Using a marker file in addition to the upgrade state to figure out how to 
handle NN restart.


Bugs: AMBARI-15482
https://issues.apache.org/jira/browse/AMBARI-15482


Repository: ambari


Description
---

During an upgrade, the HDFS NameNode(s) are restarted with the 
{{-rollingUpgrade}} flag. However, it's possible to get to the end of an 
upgrade and decide to "Finalize Later".

This allows the cluster to run in the upgraded state before committing to the 
upgrade. Full cluster control is returned via the Ambari web client.

Administrators can then decide to restart a NameNode. Upon restarting the 
NameNode, it will produce an error that it was not started with the 
{{rollingUpgrade}} flag. 

It seems that as long as an upgrade has not been finalized, the NameNode(s) 
must be started with the {{rollingUpgrade}} to allow them to function properly. 

STR:
- Perform a rolling upgrade from HDP 2.2 to 2.3 (or 2.3 to 2.4); as long as 
there is a major version change.
- Before finalization, say "Finalize Later". All services should be up and 
green.
- Now restart a NameNode


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/agent/ExecutionCommand.java
 9ea541e 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariActionExecutionHelper.java
 4ef215c 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java
 303f3a4 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java
 07061e1 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/UpgradeEntity.java
 3f1a52b 
  ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java 
ed3c772 
  
ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
 1c7ff61 
  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog222.java
 84bb9f3 
  ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql e01e693 
  ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql fb24c89 
  ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 2e092b2 
  ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 5f5df3c 
  ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql 
3d71f1f 
  ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 9d70fbb 
  ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 715000e 
  
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py
 e4c8c9c 
  
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py
 02905ec 
  
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_upgrade.py
 a154b73 
  
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py
 905802f 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderTest.java
 17c52d2 
  ambari-web/app/controllers/main/admin/stack_and_upgrade_controller.js 2dceccc 
  ambari-web/app/utils/ajax/ajax.js 29d0715 

Diff: https://reviews.apache.org/r/45035/diff/


Testing
---

Pending...


Thanks,

Jonathan Hurley

Review Request 45041: Strange paths in ambari-server setup


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45041/
---

Review request for Ambari, Andrew Onischuk, Dmitro Lisnichenko, and Dmytro Sen.


Bugs: AMBARI-15483
https://issues.apache.org/jira/browse/AMBARI-15483


Repository: ambari


Description
---

ambari-server setup command shows strange paths with double slashes in front:
Successfully installed JDK to //usr/jdk64/
Downloading JCE Policy archive from 
http://public-repo-1.hortonworks.com/ARTIFACTS/jce_policy-8.zip to 
//var/lib/ambari-server/resources/jce_policy-8.zip
Successfully downloaded JCE Policy archive to 
//var/lib/ambari-server/resources/jce_policy-8.zip
Installing JCE policy...
Completing setup...
Configuring database...
Enter advanced database configuration y/n ?
Configuring database...
Default properties detected. Using built-in database.
WARNING: Command chown None //etc/ambari-server/conf/password.dat returned exit 
code //etc/ambari-server/conf/password.dat with message: chown: invalid user: 
`None'


Diffs
-

  ambari-server/src/main/python/ambari_server/serverConfiguration.py 29187bd 

Diff: https://reviews.apache.org/r/45041/diff/


Testing
---

mvn clean test


Thanks,

Vitalyi Brodetskyi

Re: Review Request 44565: Hive view: Visual Explain does not work when there are multiple statements in the query editor

2016-03-19 Thread Pallav Kulshreshtha


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44565/
---

(Updated March 17, 2016, 4:07 p.m.)


Review request for Ambari, DIPAYAN BHOWMICK, Srimanth Gunturi, and Yusaku Sako.


Bugs: AMBARI-15346
https://issues.apache.org/jira/browse/AMBARI-15346


Repository: ambari


Description
---

Adding the "Explain | Explain Formatted" only for the last query. Make some 
modification in "updateValue" function of query editor component to handle this 
change.


Diffs (updated)
-

  
contrib/views/hive/src/main/resources/ui/hive-web/app/components/query-editor.js
 10c6bb3 
  contrib/views/hive/src/main/resources/ui/hive-web/app/controllers/index.js 
a056759 

Diff: https://reviews.apache.org/r/44565/diff/


Testing
---

manual testing done. No extra unit test case needed.


Thanks,

Pallav Kulshreshtha

Re: Review Request 44941: Report AMS / Grafana Per Disk Metrics and other changes

2016-03-19 Thread Sid Wagle


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44941/
---

(Updated March 17, 2016, 1:09 a.m.)


Review request for Ambari, Aravindan Vijayan, Dmytro Sen, and Sumit Mohanty.


Changes
---

- Added swap_in and swap_out
- Fixed memory used to reflect (used - cached)


Bugs: AMBARI-15448
https://issues.apache.org/jira/browse/AMBARI-15448


Repository: ambari


Description
---

We need to collect and expose per-disk metrics from AMS for iostats counters.
We can sort the disk names alphabetically, and name the metrics like _1, _2, 
etc.

Plus from psutils:
busy_time: (Linux, FreeBSD) time spent doing actual I/Os (in milliseconds)
read_merged_count (Linux): number of merged reads (see iostat doc)
write_merged_count (Linux): number of merged writes (see iostats doc)

_Additional fixes_:
- Combined_disk_total metric seems to be wrong : it is a sum() of all disk 
percentages from different disks
- Missing ATS and JHS metrics due to wrong config item in 
hadoop-metrics2.properties file
- Report number of disks


Diffs (updated)
-

  
ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/config_reader.py
 a053955 
  
ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/controller.py
 c04a61b 
  
ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/host_info.py 
2e8c442 
  
ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/metric_collector.py
 c28fd03 
  
ambari-metrics/ambari-metrics-host-monitoring/src/test/python/core/TestHostInfo.py
 f07b573 
  
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_grafana_util.py
 e5020a5 
  
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/templates/metric_monitor.ini.j2
 0b0932a 
  
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/templates/hadoop-metrics2.properties.j2
 47b504f 

Diff: https://reviews.apache.org/r/44941/diff/


Testing
---

Manually verified.

--
Ran 12 tests in 0.020s

OK
--
Total run:12
Total errors:0
Total failures:0
OK


Thanks,

Sid Wagle

Review Request 45054: AMBARI-15443:Make Host bulk command menu item list stack driven instead of a hardcoded list in UI code

2016-03-19 Thread Di Li


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45054/
---

Review request for Ambari, Alejandro Fernandez, Jaimin Jetly, and Yusaku Sako.


Bugs: AMBARI-15443
https://issues.apache.org/jira/browse/AMBARI-15443


Repository: ambari


Description
---

The items described here are the ones on the Hosts tab, in the Actions 
drop-down list where the UI shows entries such as All Hosts. If user mouses 
over the All Hosts, it shows a sub-list including Hosts and slave components.

The slave component items are hardcoded in hosts_table_menu_view.js as shown 
below. This jira is to put this info into each service's metainfo.xml so that 
the slave component items can be stack driven.

components: function () {
var serviceNames = App.Service.find().mapProperty('serviceName');
var menuItems = [
O.create(
{ serviceName: 'HDFS', componentName: 'DATANODE', masterComponentName: 
'NAMENODE', componentNameFormatted: 
Em.I18n.t('dashboard.services.hdfs.datanodes') }

),
O.create(
{ serviceName: 'YARN', componentName: 'NODEMANAGER', masterComponentName: 
'RESOURCEMANAGER', componentNameFormatted: 
Em.I18n.t('dashboard.services.yarn.nodeManagers') }

),
O.create(
{ serviceName: 'HBASE', componentName: 'HBASE_REGIONSERVER', 
masterComponentName: 'HBASE_MASTER', componentNameFormatted: 
Em.I18n.t('dashboard.services.hbase.regionServers') }

),
O.create(
{ serviceName: 'STORM', componentName: 'SUPERVISOR', masterComponentName: 
'SUPERVISOR', componentNameFormatted: 
Em.I18n.t('dashboard.services.storm.supervisors') }

)];

return menuItems.filter(function (item)
{ return serviceNames.contains(item.serviceName); }

);
}.property(),


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/StackServiceComponentResponse.java
 cfd4e7b 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackServiceComponentResourceProvider.java
 70945ba 
  
ambari-server/src/main/java/org/apache/ambari/server/stack/ComponentModule.java 
a122dc6 
  
ambari-server/src/main/java/org/apache/ambari/server/state/BulkCommandDefinition.java
 PRE-CREATION 
  ambari-server/src/main/java/org/apache/ambari/server/state/ComponentInfo.java 
770ee5c 
  ambari-server/src/main/resources/common-services/HAWQ/2.0.0/metainfo.xml 
e35b7d8 
  
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/metainfo.xml 
057e126 
  ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml 
75d3bea 
  ambari-server/src/main/resources/common-services/PXF/3.0.0/metainfo.xml 
afe27ec 
  ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/metainfo.xml 
804374a 
  ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/metainfo.xml 
0f71585 
  ambari-server/src/main/resources/properties.json 01c15f2 
  
ambari-server/src/test/java/org/apache/ambari/server/stack/ComponentModuleTest.java
 2f84f04 
  ambari-web/app/mappers/stack_service_mapper.js 8a65055 
  ambari-web/app/models/stack_service_component.js 26ff1b8 
  ambari-web/app/views/main/host/hosts_table_menu_view.js e75b643 
  ambari-web/test/mappers/stack_service_mapper_test.js 4bc36fe 

Diff: https://reviews.apache.org/r/45054/diff/


Testing
---

unit test
patch a trunk cluster with code change, verify the host bulk command list shown 
for DataNode, HBase Region server, storm supervisor, etc.


Thanks,

Di Li

Re: Review Request 44956: Topology host info is not cleared when a host is removed

2016-03-19 Thread Sebastian Toader


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44956/#review124168
---


Ship it!




Ship It!

- Sebastian Toader


On March 18, 2016, 10:27 a.m., Daniel Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44956/
> ---
> 
> (Updated March 18, 2016, 10:27 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Laszlo Puskas, Oliver Szabo, 
> Sandor Magyari, Sumit Mohanty, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-15457
> https://issues.apache.org/jira/browse/AMBARI-15457
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When a host is removed, a record is left in the topology_host_info table. As 
> a result re-adding the same host is not possible, due to invalid topology.
> Host removal should remove the corresponding record from topology_host_info 
> as well.
> 
> A new column with a foreign key to hosts table is added. In addition 
> ClusterTopologyImpl was statefull and hostGroupInfoMap was not updated when a 
> host was removed.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostResourceProvider.java
>  6101ee8 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/TopologyHostInfoDAO.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/TopologyHostInfoEntity.java
>  a4f251a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClustersImpl.java
>  46dd49d 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterTopology.java
>  4e178c0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterTopologyImpl.java
>  003539c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/HostGroupInfo.java
>  4847a11 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/PersistedState.java
>  dbf6735 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/PersistedStateImpl.java
>  b878955 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/TopologyManager.java
>  41f3150 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog222.java
>  8267d5d 
>   ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql dee397f 
>   ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 253b5d7 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 832d1bc 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql 
> 6c8a09d 
>   ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql a6acb74 
>   ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 2957c5c 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerTest.java
>  3e43ef1 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/HostResourceProviderTest.java
>  69c570e 
> 
> Diff: https://reviews.apache.org/r/44956/diff/
> 
> 
> Testing
> ---
> 
> Tests ran without failures. (2016-03-17 13:30)
> 
> 
> Thanks,
> 
> Daniel Gergely
> 
>

Review Request 44926: [DRAFT] Auto-retry on failure during RU/EU

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44926/
---

Review request for Ambari, Jonathan Hurley and Nate Cole.

Bugs: AMBARI-15446
https://issues.apache.org/jira/browse/AMBARI-15446

Repository: ambari

Description
---

When a failure occurs during RU/EU and the task transitions to HOLDING_FAILED
or HOLDING_TIMEDOUT, want Ambari to automatically retry up to up to x mins.
This is useful when a host goes down as Ambari is running a task on it.
ambari.properties will have 1 new parameter. E.g,.
stack-upgrade.max_retry_timeout_mins=15 (by default, will not be present)
If Ambari Server is restarted, it should be able to recover.
Today, Action Scheduler increases the attempt_count whenever a task is retried,
but it requires resetting the start_time to -1. Because of this, we cannot rely
on the start_time property to know when to timeout after several retries.

For the implementation, will add another thread to Ambari that will monitor
failed tasks only during active RU/EU and change the status back to PENDING so
that Action Scheduler can reschedule it.
Luckily, any tasks in HOLDING_TIMEDOUT and HOLDING_FAILED states are blocking,
so no other stages are allowed to proceed.
In order to know when a task was first started, will add a new property to
host_role_command table called original_start_time.

For the agents, we need to ensure that they always write out a response. On the
first heartbeat, it should send the status of its last command so we know it
failed and Ambari can retry.

Diffs
-

ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java
3a80803

ambari-server/src/main/java/org/apache/ambari/server/agent/RetryActionMonitor.java
PRE-CREATION

ambari-server/src/main/java/org/apache/ambari/server/checks/PreviousUpgradeCompleted.java
3a4467f

ambari-server/src/main/java/org/apache/ambari/server/orm/dao/ClusterVersionDAO.java
1bcca60

ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostRoleCommandDAO.java
f5b1cb4

ambari-server/src/main/java/org/apache/ambari/server/orm/entities/ClusterVersionEntity.java
f1867b4

ambari-server/src/main/java/org/apache/ambari/server/orm/entities/HostRoleCommandEntity.java
19f0602
ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java
ed3c772

ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
1c7ff61

ambari-server/src/main/java/org/apache/ambari/server/topology/LogicalRequest.java
82edbcf

Diff: https://reviews.apache.org/r/44926/diff/

Testing
---

Verified on a live cluster.

TODO: Still need to make more changes to the implementation, add the config,
switch to gauva service, add a column, and add unit tests.

Thanks,

Alejandro Fernandez

Re: Review Request 45035: Restarting HDFS Before Upgrade Finalizing Does Not Supply the rollingUpgrade Flag



> On March 18, 2016, 1:50 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariActionExecutionHelper.java,
> >  line 453
> > 
> >
> > I think adding a new column is overkill. If EU is suspended while 
> > stopping services, then HDFS has not yet been prepared or restarted with 
> > the -rollingUpgrade option.
> > 
> > I wonder if there's a way to detect on the python side that NameNode is 
> > in the middle of an upgrade and hence should be restarted with that flag.
> > Could even use a marker file that gets deleted after Finalize.

I don't think it is; it's useful for querying and allows us to deliver this 
state information via commands to any service that needs it; not just HDFS. 

The marker file is the hack approach - easily can cause problems.


- Jonathan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45035/#review124232
---


On March 18, 2016, 11:20 a.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45035/
> ---
> 
> (Updated March 18, 2016, 11:20 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Nate Cole, and Robert Levas.
> 
> 
> Bugs: AMBARI-15482
> https://issues.apache.org/jira/browse/AMBARI-15482
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> During an upgrade, the HDFS NameNode(s) are restarted with the 
> {{-rollingUpgrade}} flag. However, it's possible to get to the end of an 
> upgrade and decide to "Finalize Later".
> 
> This allows the cluster to run in the upgraded state before committing to the 
> upgrade. Full cluster control is returned via the Ambari web client.
> 
> Administrators can then decide to restart a NameNode. Upon restarting the 
> NameNode, it will produce an error that it was not started with the 
> {{rollingUpgrade}} flag. 
> 
> It seems that as long as an upgrade has not been finalized, the NameNode(s) 
> must be started with the {{rollingUpgrade}} to allow them to function 
> properly. 
> 
> STR:
> - Perform a rolling upgrade from HDP 2.2 to 2.3 (or 2.3 to 2.4); as long as 
> there is a major version change.
> - Before finalization, say "Finalize Later". All services should be up and 
> green.
> - Now restart a NameNode
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/ExecutionCommand.java
>  9ea541e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariActionExecutionHelper.java
>  4ef215c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java
>  303f3a4 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java
>  07061e1 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/UpgradeEntity.java
>  3f1a52b 
>   ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java 
> ed3c772 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
>  1c7ff61 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog222.java
>  84bb9f3 
>   ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql a85202d 
>   ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 9b4810c 
>   ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql cc3d197 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 07c786d 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql 
> ab6dc93 
>   ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 8e91fde 
>   ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 440ca44 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py
>  e4c8c9c 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py
>  02905ec 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py
>  905802f 
>   ambari-web/app/controllers/main/admin/stack_and_upgrade_controller.js 
> 2dceccc 
>   ambari-web/app/utils/ajax/ajax.js 29d0715 
> 
> Diff: https://reviews.apache.org/r/45035/diff/
> 
> 
> Testing
> ---
> 
> Pending...
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Re: Review Request 44880: AMBARI-15436 Supportability: Create 'Reset UI' button in Ambari's experimental page

2016-03-19 Thread Srimanth Gunturi


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44880/#review124098
---


Ship it!




Ship It!

- Srimanth Gunturi


On March 17, 2016, 8:52 p.m., Zhe (Joe) Wang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44880/
> ---
> 
> (Updated March 17, 2016, 8:52 p.m.)
> 
> 
> Review request for Ambari, Jaimin Jetly, Richard Zang, Srimanth Gunturi, Xi 
> Wang, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-15436
> https://issues.apache.org/jira/browse/AMBARI-15436
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Currently Ambari UI persists it's current state on the server. This helps 
> when a different user logs in, they end up in the same UI. Sometimes however, 
> UI's state persisted on server does not sync with cluster state, thereby 
> putting UI in a broken state with no recourse for the stuck user.
> 
> Ambari UI should provide a _Reset UI_ button in the experimental page which 
> resets the UI state on server.
> 
> 
> Diffs
> -
> 
>   ambari-web/app/controllers/experimental.js 9fa6da5 
>   ambari-web/app/mappers/server_data_mapper.js 1e08161 
>   ambari-web/app/messages.js 9289fc2 
>   ambari-web/app/templates/experimental.hbs 2873679 
> 
> Diff: https://reviews.apache.org/r/44880/diff/
> 
> 
> Testing
> ---
> 
> Local ambari-web test passed.
> 24578 tests complete (23 seconds)
> 145 tests pending
> Manual testing done.
> 
> 
> Thanks,
> 
> Zhe (Joe) Wang
> 
>

Re: Review Request 45012: Recommendation and validation logic for HAWQ hdfs-client.xml output.replace-datanode-on-failure property



> On March 18, 2016, 5:08 p.m., jun aoki wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py, 
> > line 699
> > 
> >
> > do you want to inline this since propertyvalue is used only once.

I prefer to leave it separate for clarity


> On March 18, 2016, 5:08 p.m., jun aoki wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py, 
> > line 984
> > 
> >
> > Do you want to set (constant) variable for a magic number 4?
> > ```
> > REPLACE_DATANODE_ON_FAILURE_THREASHOLD = 4 # my poor naming skill...
> > if numSegments > REPLACE_DATANODE_ON_FAILURE_THREASHOLD and value ... :
> >   ...
> > elif numSegments <= REPLACE_DATANODE_ON_FAILURE_THREASHOLD and value 
> > ... : 
> >   ...
> > 
> > ```
> > 
> > and you can use it on line 694

added a constant, but not for the whole file, but rather within the method. I 
feel that exposing the constant for the whole file is too intrusive.


- Alexander


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45012/#review124221
---


On March 18, 2016, 7:47 p.m., Alexander Denissov wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45012/
> ---
> 
> (Updated March 18, 2016, 7:47 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, bhuvnesh chaudhary, Goutam 
> Tadi, jun aoki, Jayush Luniya, Lav Jain, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-15499
> https://issues.apache.org/jira/browse/AMBARI-15499
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Recommendation and validation logic for HAWQ hdfs-client.xml 
> output.replace-datanode-on-failure property
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-site.xml
>  93ad3fe 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hdfs-client.xml
>  c2a756a 
>   ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 
> c0bc4e2 
>   ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 
> 4f0b4b7 
> 
> Diff: https://reviews.apache.org/r/45012/diff/
> 
> 
> Testing
> ---
> 
> manual
> 
> --
> Ran 253 tests in 7.111s
> 
> OK
> --
> Total run:927
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Alexander Denissov
> 
>

Re: Review Request 44916: StackAdvisor needs to set config map for ams-grafana-env to meet non-null expectations



> On March 16, 2016, 1:57 p.m., Sid Wagle wrote:
> > Shouldn't we also have a non-null check in the backend code?

I am not sure.. I guess it depends on the expectations. Historically it appears 
that the property map is expected to not be null.  I am happy to add null 
protections in place I can find that need to be protected.


- Robert


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44916/#review123895
---


On March 16, 2016, 12:59 p.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44916/
> ---
> 
> (Updated March 16, 2016, 12:59 p.m.)
> 
> 
> Review request for Ambari, Dmytro Sen, Jonathan Hurley, Nate Cole, and Sid 
> Wagle.
> 
> 
> Bugs: AMBARI-15442
> https://issues.apache.org/jira/browse/AMBARI-15442
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> The StackAdvisor needs to set a config map for the `ams-grafana-env`, even it 
> is empty, to meet non-null expectations in Kerberos and Blueprint logic. 
> 
> To to this, the following line needs to be added to the StackAdvisor code:
> 
> #stacks/HDP/2.0.6/services/stack_advisor.py:473
> ```
> putGrafanaProperty = self.putProperty(configurations, "ams-grafana-env", 
> services)
> ```
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
> 8c18421 
> 
> Diff: https://reviews.apache.org/r/44916/diff/
> 
> 
> Testing
> ---
> 
> Manually tested in cluster Kerbrerized using the Ambari UI.
> 
> 
> Thanks,
> 
> Robert Levas
> 
>

Re: Review Request 44835: AMBARI-15420: Refactor resource_management library


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44835/#review123892
---


Ship it!





ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py
 (line 106)


This may not necessarily be a rolling upgrade.



ambari-common/src/main/python/resource_management/libraries/script/script.py 
(line 383)


May want to ensure this is also a 3-tuple


- Alejandro Fernandez


On March 15, 2016, 8:23 p.m., Jayush Luniya wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44835/
> ---
> 
> (Updated March 15, 2016, 8:23 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Juanjo  
> Marron, Nate Cole, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-15420
> https://issues.apache.org/jira/browse/AMBARI-15420
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Refactor resource_management library, to generalize the library and remove 
> HDP-specific hardcodings
> 
> Specifically,
> 1. Make stack-selector, conf-selector tools stack-driven instead of 
> hardcoding hdp-select and conf-select
> 2. Make stack-root stack-driven instead of hardcoding /usr/hdp
> 3. Make copy_tarball mappings stack-driven 
> 4. Make PACKAGE_DIRS mapping in conf_select use stack-root instead of 
> hardcoding the "/usr/hdp"
> 
> In addition, also added a feature in the stack processing engine to support 
> properties values to be defined a external property file (See 
> tarball_map.json, stack_tools.json in patch)
> 
> Three config properties are added
> 1. cluster-env/stack_root
> 2. cluster-env/stack_tools
> 3. cluster-env/tarball_map
> Corresponding helper functions get_stack_root(), get_stack_tool(), 
> get_tarball_map() are added in script.py, which will set the defaults if 
> these config properties are not defined (ambari-server upgrade scenario needs 
> to be addressed to add these config properties on upgrade). These helper 
> functions are used to remove hardcodings in resource_management library.
> 
> 
> Remaining HDP-specific logic in resource_management library
> 
> 1. conf_select::_valid()
> 2. conf_select::get_hadoop_conf_dir()
> 3. list_ambari_managed_repos::repository_names
> 4. version_select_util::get_component_version()
> 5. script::get_stack_version()
> 6. script::should_expose_component_version()
> 7. get_lzo_packages::get_lzo_packages()
> 
> Refactoring this HDP-specific logic would require "Stack Featurization" 
> (AMBARI-13364) to be in place.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/HostCheckReportFileHandler.py 
> ee7db0a 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py
>  770595f 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py
>  647b8b6 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/default.py
>  23383dc 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/dynamic_variable_interpretation.py
>  a20b03c 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/get_stack_version.py
>  f2e6567 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/install_windows_msi.py
>  f1cd9cb 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/repo_version_history.py
>  d585dea 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/setup_ranger_plugin.py
>  4d9d8a4 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/setup_ranger_plugin_xml.py
>  2ccc0c6 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/stack_select.py
>  4a8eeb9 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/version_select_util.py
>  95c5cba 
>   
> ambari-common/src/main/python/resource_management/libraries/script/script.py 
> a8098a0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java
>  4be4049 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ConfigurationDirectory.java
>  7f21aaa 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java 
> 0c7faea 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackDefinitionDirectory.java
>  c739211 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
> 7d934bb 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/PropertyInfo.java 
> e7c9c27 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/ValueAttributesInfo.java
>

Re: Review Request 45030: Replace 0.0.0.0 with hostname in ams-site/timeline.metrics.service.webapp.address


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45030/#review124198
---


Ship it!





ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
 (line 135)


Instead of 'special' can we call this something more meaningful; like 
`BIND_ALL_IP_ADDRESS`?


- Jonathan Hurley


On March 18, 2016, 9:45 a.m., Dmytro Sen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45030/
> ---
> 
> (Updated March 18, 2016, 9:45 a.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Robert Nettleton, and Sid Wagle.
> 
> 
> Bugs: AMBARI-15480
> https://issues.apache.org/jira/browse/AMBARI-15480
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> We need collector hostname in configs. 
> Replace 0.0.0.0 with hostname in 
> ams-site/timeline.metrics.service.webapp.address
> Implement changes for blueprints, stack_advisor, upgrade catalog
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
>  e73e38f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog222.java
>  e10585e 
>   
> ambari-server/src/main/resources/common-services/ACCUMULO/1.6.1.2.2.0/package/scripts/params.py
>  3b8505b 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml
>  45c1811 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py
>  12c0f25 
>   
> ambari-server/src/main/resources/common-services/FLUME/1.4.0.2.0/package/scripts/params.py
>  5ec879c 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py
>  b517eeb 
>   
> ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/package/scripts/params.py
>  f75d578 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/params_linux.py
>  a022282 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/params.py
>  7a023cf 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
> ee2007f 
>   
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/hooks/before-START/scripts/params.py
>  469e709 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
>  ef5dc76 
>   
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog222Test.java
>  859513b 
>   ambari-server/src/test/python/stacks/2.0.6/configs/default.json fe92b5b 
>   
> ambari-server/src/test/python/stacks/2.0.6/configs/default_ams_embedded.json 
> 688dbe0 
>   
> ambari-server/src/test/python/stacks/2.0.6/configs/default_hive_non_hdfs.json 
> 4648ffc 
>   ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py 
> 9b2da28 
>   
> ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade-hdfs-secure.json
>  68afb68 
>   ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade.json 
> 79c5580 
>   ambari-server/src/test/python/stacks/2.2/configs/ranger-admin-upgrade.json 
> a7057db 
>   
> ambari-server/src/test/python/stacks/2.2/configs/ranger-usersync-upgrade.json 
> 390f9ce 
>   ambari-server/src/test/python/stacks/2.3/configs/ats_1_5.json 3fe1235 
> 
> Diff: https://reviews.apache.org/r/45030/diff/
> 
> 
> Testing
> ---
> 
> Unit tests passed
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Re: Review Request 44987: Editing tmp dir from single to multiple dir doesn't trigger the backend change

2016-03-19 Thread jun aoki



> On March 17, 2016, 11:10 p.m., Alexander Denissov wrote:
> > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/hawqsegment.py,
> >  line 45
> > 
> >
> > plural, so perhaps "..they should be.." instead of "..it should be.."

Good catch!


- jun


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44987/#review124115
---


On March 17, 2016, 10:37 p.m., jun aoki wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44987/
> ---
> 
> (Updated March 17, 2016, 10:37 p.m.)
> 
> 
> Review request for Ambari.
> 
> 
> Bugs: AMBARI-15475
> https://issues.apache.org/jira/browse/AMBARI-15475
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Modified "HAWQ Segment Temp Directory” from one [/d1/tmp/hawqsegment] to many 
> [/d1/tmp/hawqsegment,/d2/tmp/hawqsegment,/d3/tmp/hawqsegment,/d4/tmp/hawqsegment,/d5/tmp/hawqsegment]
> and the change is not propagated to segments and the folders are not created.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/common.py
>  d62332d 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/hawqsegment.py
>  0a597b6 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/master_helper.py
>  9482b35 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/params.py
>  1bdc5aa 
>   ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqmaster.py 14b5a5f 
>   ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqsegment.py 6ca83b7 
>   ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqstandby.py eaba496 
> 
> Diff: https://reviews.apache.org/r/44987/diff/
> 
> 
> Testing
> ---
> 
> manually tested.
> 
> test_configure_default (test_hawqsegment.TestHawqSegment) ... ok
> test_install_default (test_hawqsegment.TestHawqSegment) ... ok
> test_start_default (test_hawqsegment.TestHawqSegment) ... ok
> test_stop_default (test_hawqsegment.TestHawqSegment) ... ok
> test_hawq_master_critical 
> (test_alert_component_status.TestAlertComponentStatus) ... ok
> test_hawq_master_ok (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_hawq_segment_critical 
> (test_alert_component_status.TestAlertComponentStatus) ... ok
> test_hawq_segment_ok (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_hawq_standby_critical 
> (test_alert_component_status.TestAlertComponentStatus) ... ok
> test_hawq_standby_ok (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_missing_configs (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_configure_default (test_hawqstandby.TestHawqStandby) ... ok
> test_install_default (test_hawqstandby.TestHawqStandby) ... ok
> test_start_default (test_hawqstandby.TestHawqStandby) ... 2016-03-17 
> 15:33:42,969 - Standby host c6401.ambari.apache.org initialized
> ok
> test_stop_default (test_hawqstandby.TestHawqStandby) ... ok
> test_configure_default (test_hawqmaster.TestHawqMaster) ... ok
> test_install_default (test_hawqmaster.TestHawqMaster) ... ok
> test_start_default (test_hawqmaster.TestHawqMaster) ... 2016-03-17 
> 15:33:43,025 - Active master c6401.ambari.apache.org initialized
> ok
> test_start_localmaster (test_hawqmaster.TestHawqMaster) ... 2016-03-17 
> 15:33:43,049 - Master c6401.ambari.apache.org started
> ok
> test_stop_default (test_hawqmaster.TestHawqMaster) ... ok
> 
> 
> Thanks,
> 
> jun aoki
> 
>

Re: Review Request 44987: Editing tmp dir from single to multiple dir doesn't trigger the backend change

2016-03-19 Thread jun aoki


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44987/#review124128
---




ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/common.py
 (line 60)


Good point. That will acutally help cleaning some code. Will fix it.



ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/common.py
 (line 66)


```
-  for path in dir_paths.split(','):
-if path != "":
-  utils.create_dir_as_hawq_user(path)
+#   for path in dir_paths.split(','):
+# if path != "":
+#   utils.create_dir_as_hawq_user(path)
+  utils.create_dir_as_hawq_user(dir_paths)
```
I tried the code above but it does not work as commented.
When I have "/tmp/hawq/segment2,/tmp/hawq/segment3" then "segment2," (with 
a comman) was created.(I know it is weird)
I'd stick with my original.


- jun aoki


On March 17, 2016, 10:37 p.m., jun aoki wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44987/
> ---
> 
> (Updated March 17, 2016, 10:37 p.m.)
> 
> 
> Review request for Ambari.
> 
> 
> Bugs: AMBARI-15475
> https://issues.apache.org/jira/browse/AMBARI-15475
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Modified "HAWQ Segment Temp Directory” from one [/d1/tmp/hawqsegment] to many 
> [/d1/tmp/hawqsegment,/d2/tmp/hawqsegment,/d3/tmp/hawqsegment,/d4/tmp/hawqsegment,/d5/tmp/hawqsegment]
> and the change is not propagated to segments and the folders are not created.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/common.py
>  d62332d 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/hawqsegment.py
>  0a597b6 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/master_helper.py
>  9482b35 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/params.py
>  1bdc5aa 
>   ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqmaster.py 14b5a5f 
>   ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqsegment.py 6ca83b7 
>   ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqstandby.py eaba496 
> 
> Diff: https://reviews.apache.org/r/44987/diff/
> 
> 
> Testing
> ---
> 
> manually tested.
> 
> test_configure_default (test_hawqsegment.TestHawqSegment) ... ok
> test_install_default (test_hawqsegment.TestHawqSegment) ... ok
> test_start_default (test_hawqsegment.TestHawqSegment) ... ok
> test_stop_default (test_hawqsegment.TestHawqSegment) ... ok
> test_hawq_master_critical 
> (test_alert_component_status.TestAlertComponentStatus) ... ok
> test_hawq_master_ok (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_hawq_segment_critical 
> (test_alert_component_status.TestAlertComponentStatus) ... ok
> test_hawq_segment_ok (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_hawq_standby_critical 
> (test_alert_component_status.TestAlertComponentStatus) ... ok
> test_hawq_standby_ok (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_missing_configs (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_configure_default (test_hawqstandby.TestHawqStandby) ... ok
> test_install_default (test_hawqstandby.TestHawqStandby) ... ok
> test_start_default (test_hawqstandby.TestHawqStandby) ... 2016-03-17 
> 15:33:42,969 - Standby host c6401.ambari.apache.org initialized
> ok
> test_stop_default (test_hawqstandby.TestHawqStandby) ... ok
> test_configure_default (test_hawqmaster.TestHawqMaster) ... ok
> test_install_default (test_hawqmaster.TestHawqMaster) ... ok
> test_start_default (test_hawqmaster.TestHawqMaster) ... 2016-03-17 
> 15:33:43,025 - Active master c6401.ambari.apache.org initialized
> ok
> test_start_localmaster (test_hawqmaster.TestHawqMaster) ... 2016-03-17 
> 15:33:43,049 - Master c6401.ambari.apache.org started
> ok
> test_stop_default (test_hawqmaster.TestHawqMaster) ... ok
> 
> 
> Thanks,
> 
> jun aoki
> 
>

Re: Review Request 44986: AMBARI-15474: Listen for changes to auto-start configuration and send them to the agent during heartbeats


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44986/#review124222
---


Ship it!




Ship It!

- Nate Cole


On March 17, 2016, 7 p.m., Nahappan Somasundaram wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44986/
> ---
> 
> (Updated March 17, 2016, 7 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Nate Cole, Sumit Mohanty, and Sid 
> Wagle.
> 
> 
> Bugs: AMBARI-15474
> https://issues.apache.org/jira/browse/AMBARI-15474
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> AMBARI-15474: Listen for changes to auto-start configuration and send them to 
> the agent during heartbeats.
> 
> In HeartbeatHandler::handleHeartBeat(), send the recovery configuration to 
> the agent. Instead of pulling the recovery configuration from the application 
> cache, implement change events and get notified whenever there are changes to 
> the recovery configuration.
> 
> **Changes**:
> 1. Instead of sending the recovery configuration to the agent during every 
> heartbeat (10 seconds interval), send the configuration only when changes are 
> detected.
> 2. Used AmbariEventPublisher to publish the changes. RecoveryConfigHelper is 
> the subscriber which listens to changes to the configuration.
> 3. RecoveryConfigHelper maintains a map of Cluster-->Host-->Timestamp which 
> is the last updated timestamp of the recovery configuration. This timestamp 
> is used as a token and is sent to the agent with the configuration. The agent 
> provides this timestamp during each heartbeat. If the timestamp from the 
> heartbeat is the same as the one on the server side, no configuration is sent 
> to the agent.
> 4. Whenever changes are detected, the timestamp is invalidated. During the 
> next heartbeat, the recovery configuration is read from the DB and sent to 
> the agent along with the latest timestamp.
> 5. Added new unit tests to test each configuration change event.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/Controller.py 
> c1c16ac97cca0d2677da2e26eee7b4949a4bf15c 
>   ambari-agent/src/main/python/ambari_agent/Heartbeat.py 
> b28644c834a2387d2fb0ad17d104224e830a0245 
>   ambari-agent/src/main/python/ambari_agent/RecoveryManager.py 
> ed537ca6928ce376c5f3e906f7a47c3f1919e309 
>   ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeat.java 
> 1430aa20cf68db071be85c4ad2ebb9acdaccecc0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java
>  3a80803724bcb6ca9e4ec875aa17c9ac1c66fbe8 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatResponse.java
>  617b04b5b8768ba28a3c8ecb6e5e00601f153396 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/RecoveryConfig.java
>  c2c18462c586292caaf40bdb6a25a8fe9a39d76c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/RecoveryConfigHelper.java
>  92a622117c71ddfc3bd2562c04ee525491bd6ffd 
>   
> ambari-server/src/main/java/org/apache/ambari/server/events/AmbariEvent.java 
> 78731854fe80c3d6087a73ebf00b0ffe9e287354 
>   
> ambari-server/src/main/java/org/apache/ambari/server/events/ClusterConfigChangedEvent.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/events/ServiceComponentInstalledEvent.java
>  2ce01236b3bb1ab4934fe456e5669fa474987426 
>   
> ambari-server/src/main/java/org/apache/ambari/server/events/ServiceComponentRecoveryChangeEvent.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/events/ServiceComponentUninstalledEvent.java
>  19712cdb18592816f82898cd7aa08a3600c7e1f4 
>   ambari-server/src/main/java/org/apache/ambari/server/state/ConfigImpl.java 
> 2cc3d0013140c9c7d3453d7c5e05c3aa02bc5794 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/ServiceComponentImpl.java
>  4be1c21bf41416681be836716ce38305cc3f287e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java
>  3a0075e3a3c90088ca6bd2d9eca48ef50db9fdb3 
>   
> ambari-server/src/test/java/org/apache/ambari/server/agent/TestHeartbeatHandler.java
>  b1f94e38f97550123dd366984438b0bbf9c2826c 
>   
> ambari-server/src/test/java/org/apache/ambari/server/configuration/RecoveryConfigHelperTest.java
>  c86b95a7cd10b0720f0a3b9676f65a8bd68dd6c9 
>   
> ambari-server/src/test/java/org/apache/ambari/server/events/listeners/upgrade/HostVersionOutOfSyncListenerTest.java
>  1bf9d8336533bc677384314f998f10aa2f10504e 
> 
> Diff: https://reviews.apache.org/r/44986/diff/
> 
> 
> Testing
> ---
> 
> **1. mvn clean install **
> 
> 
> [INFO] 
> 
> [INFO] Reactor Summary:
>

Re: Review Request 44688: PXF should reach out to active namenode to get the delegation token on HDFS HA clusters

2016-03-19 Thread bhuvnesh chaudhary



> On March 16, 2016, 8 p.m., Jayush Luniya wrote:
> > Ship It!

Thank you Jayush. No worries, will do merge now.


- bhuvnesh


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44688/#review123919
---


On March 11, 2016, 1:26 a.m., bhuvnesh chaudhary wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44688/
> ---
> 
> (Updated March 11, 2016, 1:26 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, jun aoki, Jayush Luniya, 
> Matt, and Oleksandr Diachenko.
> 
> 
> Bugs: AMBARI-15380
> https://issues.apache.org/jira/browse/AMBARI-15380
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> PXF alerts are not working on secured HDFS HA clusters. When the cluster is 
> HA, Active Namenode should be used to get the delegation token for pxf, 
> however currently it goes to localhost.
> Updated the logic to find out the active namenode and use it.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/PXF/3.0.0/package/alerts/api_status.py
>  871da2b 
> 
> Diff: https://reviews.apache.org/r/44688/diff/
> 
> 
> Testing
> ---
> 
> yes. manually.
> 
> 
> Thanks,
> 
> bhuvnesh chaudhary
> 
>

Re: Review Request 45056: Blueprint install using config_recommendation_strategy is not functional

2016-03-19 Thread Oliver Szabo


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45056/#review124404
---


Ship it!




Ship It!

In the near future I will create a patch for fixing some issues with stack 
advisor blueprint support (e.g.: same kind of error can happen if hosts are not 
registered during stack advisor processing)

- Oliver Szabo


On March 18, 2016, 10:31 p.m., Shantanu Mundkur wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45056/
> ---
> 
> (Updated March 18, 2016, 10:31 p.m.)
> 
> 
> Review request for Ambari, Oliver Szabo and Robert Levas.
> 
> 
> Bugs: AMBARI-15454
> https://issues.apache.org/jira/browse/AMBARI-15454
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> hBlueprint install using config_recommendation strategy seems to hang for a 
> long time (couple of hours?) and ends up logging exceptions continually to 
> ambari-server.log. At the same time many hundreds of directories are seen 
> getting created under /var/run/ambari-server/stack-recommendations (I have 
> seen above 800-900). If you keep it running eventually the cluster install 
> seems to start but fails miserably at least during the start and some of it 
> makes obvious that configuration recommendations were NOT applied. You see 
> errors during startup that hint that options used for JVM (For Datanode etc) 
> were unreasonable.
> 
> Note that both the blueprint and cluster templates used empty configurations. 
> Example:
> 
> .
> .
>"configurations" : [],
>"host_groups": [
> {
>  "name": "host-group-1",
>  "configurations" : [],
>  "cardinality" : "1",
>  "components": [
>   { "name": "APP_TIMELINE_SERVER" },
>   { "name": "DATANODE" },
>   { "name": "FALCON_CLIENT" },
>   { "name": "FALCON_SERVER" },
>   { "name": "FLUME_HANDLER" },
>   { "name": "HBASE_CLIENT" },
>   { "name": "HBASE_MASTER" },
>   { "name": "HBASE_REGIONSERVER" },
> .
> .
> .
> 
> Teh cluster template was:
> { 
> "blueprint": "1node",
> "config_recommendation_strategy" : "ONLY_STACK_DEFAULTS_APPLY",
> "default_password": "myPassword1",
> "host_groups": [
> {
> "name": "host-group-1",
> "hosts": [
> {
> "fqdn": "mynode.ibm.com"
> }
> ]
> }
> ] 
> }
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
>  f5e7578 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
>  68d5755 
> 
> Diff: https://reviews.apache.org/r/45056/diff/
> 
> 
> Testing
> ---
> 
> 1) ambari-server unit tests
> 
> Results :
> 
> Tests run: 3968, Failures: 0, Errors: 0, Skipped: 33
> 2) Added unit test to 
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
>
> 3) Deployed clusters 1-5 nodes specifying "config_recommendation_strategy" : 
> "ONLY_STACK_DEFAULTS_APPLY"
> 
> 
> Thanks,
> 
> Shantanu Mundkur
> 
>

Review Request 44915: https://issues.apache.org/jira/browse/HAWQ-497


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44915/
---

Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, jun aoki, 
and Lav Jain.


Bugs: AMBARI-15435
https://issues.apache.org/jira/browse/AMBARI-15435


Repository: ambari


Description
---

Update configs for setting virtual segments in HAWQ HashDistributed Tables and 
random distributed tables


Diffs
-

  
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-site.xml
 d3aa0ed 
  ambari-server/src/main/resources/common-services/HAWQ/2.0.0/themes/theme.json 
aa35311 
  ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 
791d5b8 
  ambari-web/app/messages.js 5addc9b 

Diff: https://reviews.apache.org/r/44915/diff/


Testing
---

Yes. Manually.


Thanks,

Goutam Tadi

Re: Review Request 44916: StackAdvisor needs to set config map for ams-grafana-env to meet non-null expectations



> On March 17, 2016, 8:53 a.m., Jonathan Hurley wrote:
> > My only concern is for safety and clarify of code, does it makes sense to 
> > change the contract of the config to always return a non-null (if empty) 
> > map?

I attempted to go this route, but I got stuck where the Jackson JSON parser 
comes into play and fills the 
`org.apache.ambari.server.api.services.stackadvisor.recommendations.RecommendationResponse.BlueprintConfigurations`
 object.  This is where the null is getting set - or rather not set.  So if we 
wanted to persue this route, I can set a default value for the relevant class 
member and force it to never be null.


- Robert


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44916/#review124010
---


On March 16, 2016, 4:55 p.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44916/
> ---
> 
> (Updated March 16, 2016, 4:55 p.m.)
> 
> 
> Review request for Ambari, Dmytro Sen, Jonathan Hurley, Nate Cole, and Sid 
> Wagle.
> 
> 
> Bugs: AMBARI-15442
> https://issues.apache.org/jira/browse/AMBARI-15442
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> The StackAdvisor needs to set a config map for the `ams-grafana-env`, even it 
> is empty, to meet non-null expectations in Kerberos and Blueprint logic. 
> 
> To to this, the following line needs to be added to the StackAdvisor code:
> 
> #stacks/HDP/2.0.6/services/stack_advisor.py:473
> ```
> putGrafanaProperty = self.putProperty(configurations, "ams-grafana-env", 
> services)
> ```
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/StackAdvisorBlueprintProcessor.java
>  ea9fffb 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelperImpl.java
>  be6edc9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
>  f5e7578 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
> 8c18421 
>   
> ambari-server/src/test/java/org/apache/ambari/server/api/services/stackadvisor/StackAdvisorBlueprintProcessorTest.java
>  c4e694b 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/KerberosHelperTest.java
>  f6027f3 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
>  68d5755 
>   ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py 
> 9070c0a 
> 
> Diff: https://reviews.apache.org/r/44916/diff/
> 
> 
> Testing
> ---
> 
> Manually tested in cluster Kerbrerized using the Ambari UI.
> 
> 
> Thanks,
> 
> Robert Levas
> 
>

Re: Review Request 44868: Changed HAWQ directory descriptions


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44868/#review123918
---


Ship it!




Ship It!

- Jayush Luniya


On March 15, 2016, 10:51 p.m., Alexander Denissov wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44868/
> ---
> 
> (Updated March 15, 2016, 10:51 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, bhuvnesh chaudhary, Goutam 
> Tadi, jun aoki, Jayush Luniya, Lav Jain, Oleksandr Diachenko, and Sumit 
> Mohanty.
> 
> 
> Bugs: AMBARI-15433
> https://issues.apache.org/jira/browse/AMBARI-15433
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Changed HAWQ directory descriptions
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-site.xml
>  d3aa0ed 
> 
> Diff: https://reviews.apache.org/r/44868/diff/
> 
> 
> Testing
> ---
> 
> manual.
> 
> 
> Thanks,
> 
> Alexander Denissov
> 
>

Re: Review Request 44987: Editing tmp dir from single to multiple dir doesn't trigger the backend change

2016-03-19 Thread bhuvnesh chaudhary


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44987/#review124309
---


Ship it!




Ship It!

- bhuvnesh chaudhary


On March 18, 2016, 10:17 p.m., jun aoki wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44987/
> ---
> 
> (Updated March 18, 2016, 10:17 p.m.)
> 
> 
> Review request for Ambari.
> 
> 
> Bugs: AMBARI-15475
> https://issues.apache.org/jira/browse/AMBARI-15475
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Modified "HAWQ Segment Temp Directory” from one [/d1/tmp/hawqsegment] to many 
> [/d1/tmp/hawqsegment,/d2/tmp/hawqsegment,/d3/tmp/hawqsegment,/d4/tmp/hawqsegment,/d5/tmp/hawqsegment]
> and the change is not propagated to segments and the folders are not created.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/common.py
>  d62332d 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/hawqsegment.py
>  0a597b6 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/master_helper.py
>  9482b35 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/params.py
>  1bdc5aa 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/utils.py
>  dc6c203 
>   ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqmaster.py 14b5a5f 
>   ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqsegment.py 6ca83b7 
>   ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqstandby.py eaba496 
> 
> Diff: https://reviews.apache.org/r/44987/diff/
> 
> 
> Testing
> ---
> 
> manually tested.
> 
> test_configure_default (test_hawqsegment.TestHawqSegment) ... ok
> test_install_default (test_hawqsegment.TestHawqSegment) ... ok
> test_start_default (test_hawqsegment.TestHawqSegment) ... ok
> test_stop_default (test_hawqsegment.TestHawqSegment) ... ok
> test_hawq_master_critical 
> (test_alert_component_status.TestAlertComponentStatus) ... ok
> test_hawq_master_ok (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_hawq_segment_critical 
> (test_alert_component_status.TestAlertComponentStatus) ... ok
> test_hawq_segment_ok (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_hawq_standby_critical 
> (test_alert_component_status.TestAlertComponentStatus) ... ok
> test_hawq_standby_ok (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_missing_configs (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_configure_default (test_hawqstandby.TestHawqStandby) ... ok
> test_install_default (test_hawqstandby.TestHawqStandby) ... ok
> test_start_default (test_hawqstandby.TestHawqStandby) ... 2016-03-17 
> 15:33:42,969 - Standby host c6401.ambari.apache.org initialized
> ok
> test_stop_default (test_hawqstandby.TestHawqStandby) ... ok
> test_configure_default (test_hawqmaster.TestHawqMaster) ... ok
> test_install_default (test_hawqmaster.TestHawqMaster) ... ok
> test_start_default (test_hawqmaster.TestHawqMaster) ... 2016-03-17 
> 15:33:43,025 - Active master c6401.ambari.apache.org initialized
> ok
> test_start_localmaster (test_hawqmaster.TestHawqMaster) ... 2016-03-17 
> 15:33:43,049 - Master c6401.ambari.apache.org started
> ok
> test_stop_default (test_hawqmaster.TestHawqMaster) ... ok
> 
> 
> Thanks,
> 
> jun aoki
> 
>

Re: Review Request 44958: Increase Ambari Server Perm gen default value


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44958/#review124042
---


Fix it, then Ship it!





ambari-server/src/main/python/ambari_server_main.py (line 89)


I am not familiar with these options, but it apears like no one really 
knows what they do.  See https://dzone.com/articles/busting-permgen-myths for 
an example. 

Do we really think that this is necessary?  Do we know how this will affect 
performance?


- Robert Levas


On March 17, 2016, 10:57 a.m., Daniel Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44958/
> ---
> 
> (Updated March 17, 2016, 10:57 a.m.)
> 
> 
> Review request for Ambari, Laszlo Puskas, Oliver Szabo, Robert Levas, Sandor 
> Magyari, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-15465
> https://issues.apache.org/jira/browse/AMBARI-15465
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Increased permGen size to 128m, also added **-XX:+CMSPermGenSweepingEnabled** 
> and **-XX:+CMSClassUnloadingEnabled** to allow jvm to keep permGen space 
> clean.
> 
> 
> Diffs
> -
> 
>   ambari-server/conf/unix/ambari-env.sh e9fdcdd 
>   ambari-server/src/main/python/ambari_server_main.py 00e084b 
> 
> Diff: https://reviews.apache.org/r/44958/diff/
> 
> 
> Testing
> ---
> 
> Manual testing:
> [root@c6401 sbin]# ps -ef | grep ambari-server
> root 20769 1 27 14:47 pts/000:00:24 
> //usr/jdk64/jdk1.8.0_60/bin/java -server -XX:NewRatio=3 
> -XX:+UseConcMarkSweepGC -XX:-UseGCOverheadLimit 
> -XX:CMSInitiatingOccupancyFraction=60 **-XX:+CMSPermGenSweepingEnabled 
> -XX:+CMSClassUnloadingEnabled** -Dsun.zip.disableMemoryMapping=true -Xms512m 
> -Xmx2048m **-XX:MaxPermSize=128m** 
> -Djava.security.auth.login.config=//etc/ambari-server/conf/krb5JAASLogin.conf 
> -Djava.security.krb5.conf=/etc/krb5.conf 
> -Djavax.security.auth.useSubjectCredsOnly=false -Xms512m -Xmx2048m 
> -Djava.security.auth.login.config=//etc/ambari-server/conf/krb5JAASLogin.conf 
> -Djava.security.krb5.conf=/etc/krb5.conf 
> -Djavax.security.auth.useSubjectCredsOnly=false -cp 
> //etc/ambari-server/conf:/usr/lib/ambari-server/*:/usr/share/java/postgresql-jdbc.jar
>  org.apache.ambari.server.controller.AmbariServer
> 
> 
> Thanks,
> 
> Daniel Gergely
> 
>

Review Request 44931: Use Version Definition value for package-version when installing


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44931/
---

Review request for Ambari, Alejandro Fernandez and Jonathan Hurley.


Bugs: AMBARI-15447
https://issues.apache.org/jira/browse/AMBARI-15447


Repository: ambari


Description
---

The version defintion file defines a package version that should be used when 
installing packages.  The current calculation cannot be considered reliable, 
but should stay as a contingency for when NOT using a version definition.


Diffs
-

  ambari-common/src/main/python/resource_management/libraries/script/script.py 
a8098a0 
  
ambari-server/src/main/java/org/apache/ambari/server/agent/ExecutionCommand.java
 402a338 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
 e32e2f9 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProvider.java
 07e62b3 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostStackVersionResourceProvider.java
 1cd9c0a 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RepositoryVersionEntity.java
 3398709 
  
ambari-server/src/main/java/org/apache/ambari/server/state/repository/Release.java
 6bcedf5 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProviderTest.java
 dea83a1 
  ambari-server/src/test/python/custom_actions/TestInstallPackages.py f022c80 
  ambari-server/src/test/resources/hbase_version_test.xml 9df07ed 

Diff: https://reviews.apache.org/r/44931/diff/


Testing
---

Manual.  Automated pending.


Thanks,

Nate Cole

Review Request 44916: StackAdvisor needs to set config map for ams-grafana-env to meet non-null expectations


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44916/
---

Review request for Ambari, Dmytro Sen, Jonathan Hurley, Nate Cole, and Sid 
Wagle.


Bugs: AMBARI-15442
https://issues.apache.org/jira/browse/AMBARI-15442


Repository: ambari


Description
---

The StackAdvisor needs to set a config map for the `ams-grafana-env`, even it 
is empty, to meet non-null expectations in Kerberos and Blueprint logic. 

To to this, the following line needs to be added to the StackAdvisor code:

#stacks/HDP/2.0.6/services/stack_advisor.py:473
```
putGrafanaProperty = self.putProperty(configurations, "ams-grafana-env", 
services)
```


Diffs
-

  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
8c18421 

Diff: https://reviews.apache.org/r/44916/diff/


Testing
---

Manually tested in cluster Kerbrerized using the Ambari UI.


Thanks,

Robert Levas

Re: Review Request 45035: Restarting HDFS Before Upgrade Finalizing Does Not Supply the rollingUpgrade Flag


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45035/#review124232
---




ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariActionExecutionHelper.java
 (line 441)


I think adding a new column is overkill. If EU is suspended while stopping 
services, then HDFS has not yet been prepared or restarted with the 
-rollingUpgrade option.

I wonder if there's a way to detect on the python side that NameNode is in 
the middle of an upgrade and hence should be restarted with that flag.
Could even use a marker file that gets deleted after Finalize.


- Alejandro Fernandez


On March 18, 2016, 3:20 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45035/
> ---
> 
> (Updated March 18, 2016, 3:20 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Nate Cole, and Robert Levas.
> 
> 
> Bugs: AMBARI-15482
> https://issues.apache.org/jira/browse/AMBARI-15482
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> During an upgrade, the HDFS NameNode(s) are restarted with the 
> {{-rollingUpgrade}} flag. However, it's possible to get to the end of an 
> upgrade and decide to "Finalize Later".
> 
> This allows the cluster to run in the upgraded state before committing to the 
> upgrade. Full cluster control is returned via the Ambari web client.
> 
> Administrators can then decide to restart a NameNode. Upon restarting the 
> NameNode, it will produce an error that it was not started with the 
> {{rollingUpgrade}} flag. 
> 
> It seems that as long as an upgrade has not been finalized, the NameNode(s) 
> must be started with the {{rollingUpgrade}} to allow them to function 
> properly. 
> 
> STR:
> - Perform a rolling upgrade from HDP 2.2 to 2.3 (or 2.3 to 2.4); as long as 
> there is a major version change.
> - Before finalization, say "Finalize Later". All services should be up and 
> green.
> - Now restart a NameNode
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/ExecutionCommand.java
>  9ea541e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariActionExecutionHelper.java
>  4ef215c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java
>  303f3a4 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java
>  07061e1 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/UpgradeEntity.java
>  3f1a52b 
>   ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java 
> ed3c772 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
>  1c7ff61 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog222.java
>  84bb9f3 
>   ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql a85202d 
>   ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 9b4810c 
>   ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql cc3d197 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 07c786d 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql 
> ab6dc93 
>   ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 8e91fde 
>   ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 440ca44 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py
>  e4c8c9c 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py
>  02905ec 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py
>  905802f 
>   ambari-web/app/controllers/main/admin/stack_and_upgrade_controller.js 
> 2dceccc 
>   ambari-web/app/utils/ajax/ajax.js 29d0715 
> 
> Diff: https://reviews.apache.org/r/45035/diff/
> 
> 
> Testing
> ---
> 
> Pending...
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Review Request 44972: Improve error logging for install errors during blueprint deployments.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44972/
---

Review request for Ambari, Di Li and Sumit Mohanty.


Bugs: AMBARI-15412
https://issues.apache.org/jira/browse/AMBARI-15412


Repository: ambari


Description
---

Improve error logging for install errors during blueprint deployments. 

Currently a severe error during install of a service component gets logged as a 
WARNing.

E.g.:

09 Mar 2016 12:11:45,881 WARN [qtp-ambari-agent-146] HeartBeatHandler:603 - 
Operation failed - may be retried. Service component host: KAFKA_BROKER, host: 
hdtest159.svl.ibm.com Action id12-0


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java
 24fea22 

Diff: https://reviews.apache.org/r/44972/diff/


Testing
---

There are no JUnit test cases, but attached the screenshot which shows modified 
output.


File Attachments


Patch
  
https://reviews.apache.org/media/uploaded/files/2016/03/17/2110d517-0cf4-4477-87e7-935c963e04de__AMBARI-15412.patch
Output
  
https://reviews.apache.org/media/uploaded/files/2016/03/17/923d218f-7ca4-4439-b42f-743511936f94__AMBARI-15412_output.png


Thanks,

Amruta Borkar

Review Request 45030: Replace 0.0.0.0 with hostname in ams-site/timeline.metrics.service.webapp.address

2016-03-19 Thread Dmytro Sen


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45030/
---

Review request for Ambari, Jonathan Hurley, Robert Nettleton, and Sid Wagle.


Bugs: AMBARI-15480
https://issues.apache.org/jira/browse/AMBARI-15480


Repository: ambari


Description
---

We need collector hostname in configs. 
Replace 0.0.0.0 with hostname in 
ams-site/timeline.metrics.service.webapp.address
Implement changes for blueprints, stack_advisor, upgrade catalog


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
 e73e38f 
  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog222.java
 e10585e 
  
ambari-server/src/main/resources/common-services/ACCUMULO/1.6.1.2.2.0/package/scripts/params.py
 3b8505b 
  
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml
 45c1811 
  
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py
 12c0f25 
  
ambari-server/src/main/resources/common-services/FLUME/1.4.0.2.0/package/scripts/params.py
 5ec879c 
  
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py
 b517eeb 
  
ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/package/scripts/params.py
 f75d578 
  
ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/params_linux.py
 a022282 
  
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/params.py
 7a023cf 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
ee2007f 
  
ambari-server/src/main/resources/stacks/HDPWIN/2.1/hooks/before-START/scripts/params.py
 469e709 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
 ef5dc76 
  
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog222Test.java
 859513b 
  ambari-server/src/test/python/stacks/2.0.6/configs/default.json fe92b5b 
  ambari-server/src/test/python/stacks/2.0.6/configs/default_ams_embedded.json 
688dbe0 
  ambari-server/src/test/python/stacks/2.0.6/configs/default_hive_non_hdfs.json 
4648ffc 
  ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py 9b2da28 
  
ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade-hdfs-secure.json
 68afb68 
  ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade.json 
79c5580 
  ambari-server/src/test/python/stacks/2.2/configs/ranger-admin-upgrade.json 
a7057db 
  ambari-server/src/test/python/stacks/2.2/configs/ranger-usersync-upgrade.json 
390f9ce 
  ambari-server/src/test/python/stacks/2.3/configs/ats_1_5.json 3fe1235 

Diff: https://reviews.apache.org/r/45030/diff/


Testing
---

Unit tests passed


Thanks,

Dmytro Sen

Re: Review Request 44687: Update RCO : PXF should start after HDFS

2016-03-19 Thread Tim Thorpe



> On March 11, 2016, 9:06 a.m., Jayush Luniya wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.3/role_command_order.json, 
> > line 14
> > 
> >
> > RCO can be defined at service level 
> > (https://issues.apache.org/jira/browse/AMBARI-9363). Instead of updating 
> > RCO of HDP stack, can you move the HAWQ and PXF specific ordering into the 
> > service?
> 
> bhuvnesh chaudhary wrote:
> Thanks Jayush for the suggestion, let me try that.
> However, RCO at service level will be applicable only for trunk, for 2.2 
> branch we still need to update HDP rco. correct ? 
> If so i will remove trunk tag from the list of branches, and keep this 
> pathc for 2.2.0 only.
> 
> bhuvnesh chaudhary wrote:
> Tried putting role_command_order.json under 
> /var/lib/ambari-server/resources/common-services/HAWQ/2.0.0/role_command_order.json.
>  However, it appears that its not loaded due to which HAWQMASTER was started 
> before NAMENODE and it failed. Same for PXF.
> Here are the contents of RCO: 
> [root@c6401 ~]# cat 
> /var/lib/ambari-server/resources/common-services/HAWQ/2.0.0/role_command_order.json
> {
>   "_comment" : "Record format:",
>   "_comment" : "blockedRole-blockedCommand: 
> [blockerRole1-blockerCommand1, blockerRole2-blockerCommand2, ...]",
>   "general_deps" : {
> "_comment" : "dependencies for all cases",
> "HAWQMASTER-START" : ["NAMENODE-START", "DATANODE-START", 
> "NODEMANAGER-START"],
> "HAWQSTANDBY-START" : ["HAWQMASTER-START"],
> "HAWQSEGMENT-START" : ["HAWQMASTER-START", "HAWQSTANDBY-START"],
> "HAWQ_SERVICE_CHECK-SERVICE_CHECK" : ["HAWQSEGMENT-START", 
> "HDFS_SERVICE_CHECK-SERVICE_CHECK", "YARN_SERVICE_CHECK-SERVICE_CHECK", 
> "PXF_SERVICE_CHECK-SERVICE_CHECK"]
>   }
> }
> 
> Checked ambari-server.log, it does not show that this RCO is loaded but 
> shows the others under /var/lib/ambari-server/resources/stacks which are 
> loaded, will need to debug further for the root cause. 
> Adding Tim Thorpe as well to the reviewers list.
> 
> Jayush Luniya wrote:
> @Bhuvnesh
> Go ahead with your changes without the refactoring in trunk and 2.2 
> branch. I dont want to block you on it. Lets handle the refactoring in a 
> separate JIRA. @Tim Thorpe, can you check if anything is wrong with the 
> service-level RCO?
> 
> bhuvnesh chaudhary wrote:
> Thank you Jayush, will go ahead with the change.

When I added the RCO change for putting it at the service level, I only tested 
with services defined in the stack not having inherited the RCO from 
common-services.  I'll look into it some more and see if I can figure out what 
the problem might be.  Sorry was on vacation for a week.


- Tim


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44687/#review123118
---


On March 11, 2016, 9:50 p.m., bhuvnesh chaudhary wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44687/
> ---
> 
> (Updated March 11, 2016, 9:50 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, jun aoki, Jayush Luniya, 
> Matt, Oleksandr Diachenko, and Tim Thorpe.
> 
> 
> Bugs: AMBARI-15381
> https://issues.apache.org/jira/browse/AMBARI-15381
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Update RCO to ensure that PXF is started after HDFS. 
> PXF connects to namenode during startup and retries couple of times until 
> namenode has come up.
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/stacks/HDP/2.3/role_command_order.json 
> b28f2a9 
> 
> Diff: https://reviews.apache.org/r/44687/diff/
> 
> 
> Testing
> ---
> 
> yes. manually.
> 
> 
> Thanks,
> 
> bhuvnesh chaudhary
> 
>

Re: Review Request 44987: Editing tmp dir from single to multiple dir doesn't trigger the backend change

2016-03-19 Thread bhuvnesh chaudhary


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44987/#review124306
---




ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/common.py
 (line 57)


with hawq you don't call master directory for segment hawq_segment_dir



ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/common.py
 (line 67)


probably should have stripped of spaces before creating the dir.


- bhuvnesh chaudhary


On March 18, 2016, 10:17 p.m., jun aoki wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44987/
> ---
> 
> (Updated March 18, 2016, 10:17 p.m.)
> 
> 
> Review request for Ambari.
> 
> 
> Bugs: AMBARI-15475
> https://issues.apache.org/jira/browse/AMBARI-15475
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Modified "HAWQ Segment Temp Directory” from one [/d1/tmp/hawqsegment] to many 
> [/d1/tmp/hawqsegment,/d2/tmp/hawqsegment,/d3/tmp/hawqsegment,/d4/tmp/hawqsegment,/d5/tmp/hawqsegment]
> and the change is not propagated to segments and the folders are not created.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/common.py
>  d62332d 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/hawqsegment.py
>  0a597b6 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/master_helper.py
>  9482b35 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/params.py
>  1bdc5aa 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/utils.py
>  dc6c203 
>   ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqmaster.py 14b5a5f 
>   ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqsegment.py 6ca83b7 
>   ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqstandby.py eaba496 
> 
> Diff: https://reviews.apache.org/r/44987/diff/
> 
> 
> Testing
> ---
> 
> manually tested.
> 
> test_configure_default (test_hawqsegment.TestHawqSegment) ... ok
> test_install_default (test_hawqsegment.TestHawqSegment) ... ok
> test_start_default (test_hawqsegment.TestHawqSegment) ... ok
> test_stop_default (test_hawqsegment.TestHawqSegment) ... ok
> test_hawq_master_critical 
> (test_alert_component_status.TestAlertComponentStatus) ... ok
> test_hawq_master_ok (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_hawq_segment_critical 
> (test_alert_component_status.TestAlertComponentStatus) ... ok
> test_hawq_segment_ok (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_hawq_standby_critical 
> (test_alert_component_status.TestAlertComponentStatus) ... ok
> test_hawq_standby_ok (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_missing_configs (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_configure_default (test_hawqstandby.TestHawqStandby) ... ok
> test_install_default (test_hawqstandby.TestHawqStandby) ... ok
> test_start_default (test_hawqstandby.TestHawqStandby) ... 2016-03-17 
> 15:33:42,969 - Standby host c6401.ambari.apache.org initialized
> ok
> test_stop_default (test_hawqstandby.TestHawqStandby) ... ok
> test_configure_default (test_hawqmaster.TestHawqMaster) ... ok
> test_install_default (test_hawqmaster.TestHawqMaster) ... ok
> test_start_default (test_hawqmaster.TestHawqMaster) ... 2016-03-17 
> 15:33:43,025 - Active master c6401.ambari.apache.org initialized
> ok
> test_start_localmaster (test_hawqmaster.TestHawqMaster) ... 2016-03-17 
> 15:33:43,049 - Master c6401.ambari.apache.org started
> ok
> test_stop_default (test_hawqmaster.TestHawqMaster) ... ok
> 
> 
> Thanks,
> 
> jun aoki
> 
>

Re: Review Request 44958: Increase Ambari Server Perm gen default value

2016-03-19 Thread Daniel Gergely



> On márc. 17, 2016, 4:37 du, Robert Levas wrote:
> > ambari-server/src/main/python/ambari_server_main.py, line 89
> > 
> >
> > I am not familiar with these options, but it apears like no one really 
> > knows what they do.  See https://dzone.com/articles/busting-permgen-myths 
> > for an example. 
> > 
> > Do we really think that this is necessary?  Do we know how this will 
> > affect performance?

These options are for JVM, it can be checked in the jvm implementors webpage. 
For example IBM does not use permgen space at all, but oracle does. The first 
option has not effect from java 7, but it was useful in java 6 and before, so I 
removed that one.
However -XX:+CMSClassUnloadingEnabled is useful. When classes are loaded 
(statically or dynamically) they are stored in permgen space and remains there 
forever (well, almost...). Since we use CMS (-XX:+UseConcMarkSweepGC flag is 
already defined), we can allow GC to unload unused classes. As a result it can 
increase the free space in permGen. In normal case when permGen space is full, 
garbage collector initiates a STW (stop-the-world) action to clean permgen 
space.
Short description here: 
https://blogs.oracle.com/poonam/entry/about_g1_garbage_collector_permanent
Oracle tutorial here: 
http://www.oracle.com/technetwork/tutorials/tutorials-1876574.html

>From Java 8 as I know there is no permGen space, so this issue relates only to 
>java 7.

So I removed -XX:+CMSPermGenSweepingEnabled, but I kept 
-XX:+CMSClassUnloadingEnabled. Performance wise you should not experience any 
difference.


- Daniel


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44958/#review124042
---


On márc. 17, 2016, 2:57 du, Daniel Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44958/
> ---
> 
> (Updated márc. 17, 2016, 2:57 du)
> 
> 
> Review request for Ambari, Laszlo Puskas, Oliver Szabo, Robert Levas, Sandor 
> Magyari, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-15465
> https://issues.apache.org/jira/browse/AMBARI-15465
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Increased permGen size to 128m, also added **-XX:+CMSPermGenSweepingEnabled** 
> and **-XX:+CMSClassUnloadingEnabled** to allow jvm to keep permGen space 
> clean.
> 
> 
> Diffs
> -
> 
>   ambari-server/conf/unix/ambari-env.sh e9fdcdd 
>   ambari-server/src/main/python/ambari_server_main.py 00e084b 
> 
> Diff: https://reviews.apache.org/r/44958/diff/
> 
> 
> Testing
> ---
> 
> Manual testing:
> [root@c6401 sbin]# ps -ef | grep ambari-server
> root 20769 1 27 14:47 pts/000:00:24 
> //usr/jdk64/jdk1.8.0_60/bin/java -server -XX:NewRatio=3 
> -XX:+UseConcMarkSweepGC -XX:-UseGCOverheadLimit 
> -XX:CMSInitiatingOccupancyFraction=60 **-XX:+CMSPermGenSweepingEnabled 
> -XX:+CMSClassUnloadingEnabled** -Dsun.zip.disableMemoryMapping=true -Xms512m 
> -Xmx2048m **-XX:MaxPermSize=128m** 
> -Djava.security.auth.login.config=//etc/ambari-server/conf/krb5JAASLogin.conf 
> -Djava.security.krb5.conf=/etc/krb5.conf 
> -Djavax.security.auth.useSubjectCredsOnly=false -Xms512m -Xmx2048m 
> -Djava.security.auth.login.config=//etc/ambari-server/conf/krb5JAASLogin.conf 
> -Djava.security.krb5.conf=/etc/krb5.conf 
> -Djavax.security.auth.useSubjectCredsOnly=false -cp 
> //etc/ambari-server/conf:/usr/lib/ambari-server/*:/usr/share/java/postgresql-jdbc.jar
>  org.apache.ambari.server.controller.AmbariServer
> 
> 
> Thanks,
> 
> Daniel Gergely
> 
>

Re: Review Request 44931: Use Version Definition value for package-version when installing


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44931/
---

(Updated March 16, 2016, 6:08 p.m.)


Review request for Ambari, Alejandro Fernandez and Jonathan Hurley.


Bugs: AMBARI-15447
https://issues.apache.org/jira/browse/AMBARI-15447


Repository: ambari


Description
---

The version defintion file defines a package version that should be used when 
installing packages.  The current calculation cannot be considered reliable, 
but should stay as a contingency for when NOT using a version definition.


Diffs
-

  ambari-common/src/main/python/resource_management/libraries/script/script.py 
a8098a0 
  
ambari-server/src/main/java/org/apache/ambari/server/agent/ExecutionCommand.java
 402a338 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
 e32e2f9 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProvider.java
 07e62b3 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostStackVersionResourceProvider.java
 1cd9c0a 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RepositoryVersionEntity.java
 3398709 
  
ambari-server/src/main/java/org/apache/ambari/server/state/repository/Release.java
 6bcedf5 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProviderTest.java
 dea83a1 
  ambari-server/src/test/python/custom_actions/TestInstallPackages.py f022c80 
  ambari-server/src/test/resources/hbase_version_test.xml 9df07ed 

Diff: https://reviews.apache.org/r/44931/diff/


Testing (updated)
---

Manual.  Automated:

Tests run: 3961, Failures: 0, Errors: 0, Skipped: 33

[INFO] 
[INFO] BUILD FAILURE
[INFO] 
[INFO] Total time: 43:00.985s
[INFO] Finished at: Wed Mar 16 17:50:32 EDT 2016
[INFO] Final Memory: 35M/776M
[INFO] 


Thanks,

Nate Cole

Re: Review Request 44835: AMBARI-15420: Refactor resource_management library


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44835/
---

(Updated March 18, 2016, 8:57 p.m.)


Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Juanjo  
Marron, Nate Cole, Sumit Mohanty, and Sid Wagle.


Bugs: AMBARI-15420
https://issues.apache.org/jira/browse/AMBARI-15420


Repository: ambari


Description
---

Refactor resource_management library, to generalize the library and remove 
HDP-specific hardcodings

Specifically,
1. Make stack-selector, conf-selector tools stack-driven instead of hardcoding 
hdp-select and conf-select
2. Make stack-root stack-driven instead of hardcoding /usr/hdp
3. Make copy_tarball mappings stack-driven 
4. Make PACKAGE_DIRS mapping in conf_select use stack-root instead of 
hardcoding the "/usr/hdp"

In addition, also added a feature in the stack processing engine to support 
properties values to be defined a external property file (See tarball_map.json, 
stack_tools.json in patch)

Three config properties are added
1. cluster-env/stack_root
2. cluster-env/stack_tools
3. cluster-env/tarball_map
Corresponding helper functions get_stack_root(), get_stack_tool(), 
get_tarball_map() are added in script.py, which will set the defaults if these 
config properties are not defined (ambari-server upgrade scenario needs to be 
addressed to add these config properties on upgrade). These helper functions 
are used to remove hardcodings in resource_management library.


Remaining HDP-specific logic in resource_management library

1. conf_select::_valid()
2. conf_select::get_hadoop_conf_dir()
3. list_ambari_managed_repos::repository_names
4. version_select_util::get_component_version()
5. script::get_stack_version()
6. script::should_expose_component_version()
7. get_lzo_packages::get_lzo_packages()

Refactoring this HDP-specific logic would require "Stack Featurization" 
(AMBARI-13364) to be in place.


Diffs
-

  ambari-agent/src/main/python/ambari_agent/HostCheckReportFileHandler.py 
ee7db0a 
  ambari-common/src/main/python/resource_management/core/utils.py 71d0008 
  
ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py
 770595f 
  
ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py
 647b8b6 
  
ambari-common/src/main/python/resource_management/libraries/functions/default.py
 23383dc 
  
ambari-common/src/main/python/resource_management/libraries/functions/dynamic_variable_interpretation.py
 a20b03c 
  
ambari-common/src/main/python/resource_management/libraries/functions/get_stack_version.py
 f2e6567 
  
ambari-common/src/main/python/resource_management/libraries/functions/install_windows_msi.py
 f1cd9cb 
  
ambari-common/src/main/python/resource_management/libraries/functions/repo_version_history.py
 d585dea 
  
ambari-common/src/main/python/resource_management/libraries/functions/setup_ranger_plugin.py
 4d9d8a4 
  
ambari-common/src/main/python/resource_management/libraries/functions/setup_ranger_plugin_xml.py
 2ccc0c6 
  
ambari-common/src/main/python/resource_management/libraries/functions/stack_select.py
 4a8eeb9 
  
ambari-common/src/main/python/resource_management/libraries/functions/stack_tools.py
 PRE-CREATION 
  
ambari-common/src/main/python/resource_management/libraries/functions/version_select_util.py
 95c5cba 
  ambari-common/src/main/python/resource_management/libraries/script/script.py 
1b0116e 
  
ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java
 4be4049 
  
ambari-server/src/main/java/org/apache/ambari/server/stack/ConfigurationDirectory.java
 7f21aaa 
  ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java 
0c7faea 
  
ambari-server/src/main/java/org/apache/ambari/server/stack/StackDefinitionDirectory.java
 c739211 
  ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
7d934bb 
  ambari-server/src/main/java/org/apache/ambari/server/state/PropertyInfo.java 
e7c9c27 
  
ambari-server/src/main/java/org/apache/ambari/server/state/ValueAttributesInfo.java
 3f7f756 
  ambari-server/src/main/java/org/apache/ambari/server/utils/JsonUtils.java 
PRE-CREATION 
  
ambari-server/src/main/resources/common-services/ACCUMULO/1.6.1.2.2.0/package/scripts/accumulo_script.py
 12ca388 
  
ambari-server/src/main/resources/common-services/ACCUMULO/1.6.1.2.2.0/package/scripts/status_params.py
 45dbb24 
  
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/atlas_client.py
 d000846 
  
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/status_params.py
 2c06c40 
  
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/status_params.py
 535c821 
  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hcat_client.py
 85e7012

Review Request 44957: [Ambari tarballs] ambari-server setup fails

2016-03-19 Thread Andrew Onischuk


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44957/
---

Review request for Ambari and Vitalyi Brodetskyi.


Bugs: AMBARI-15464
https://issues.apache.org/jira/browse/AMBARI-15464


Repository: ambari


Description
---

[ambari@c6401 ~]$ /opt/usr/sbin/ambari-server status
Using python  /usr/bin/python
Ambari-server status
Traceback (most recent call last):
  File "/opt/usr/sbin/ambari-server.py", line 33, in 
from ambari_server.dbConfiguration import DATABASE_NAMES, 
LINUX_DBMS_KEYS_LIST
  File 
"/opt/usr/lib/python2.6/site-packages/ambari_server/dbConfiguration.py", line 
28, in 
from ambari_server.serverConfiguration import 
decrypt_password_for_alias, get_ambari_properties, get_is_secure, \
  File 
"/opt/usr/lib/python2.6/site-packages/ambari_server/serverConfiguration.py", 
line 517, in 
configDefaults = ServerConfigDefaults()
  File 
"/opt/usr/lib/python2.6/site-packages/ambari_server/serverConfiguration.py", 
line 438, in __init__
super(ServerConfigDefaultsLinux, self).__init__()
  File 
"/opt/usr/lib/python2.6/site-packages/ambari_server/serverConfiguration.py", 
line 328, in __init__
self.OUT_DIR = 
parse_log4j_file("/etc/ambari-server/conf/log4j.properties")['ambari.log.dir']
  File "/opt/usr/lib/python2.6/site-packages/ambari_commons/os_utils.py", 
line 188, in parse_log4j_file
with open(filename, "rb") as fp:
IOError: [Errno 2] No such file or directory: 
'/etc/ambari-server/conf/log4j.properties'


Diffs
-

  ambari-server/src/main/python/ambari_server/serverConfiguration.py dc1fd8a 

Diff: https://reviews.apache.org/r/44957/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Re: Review Request 44878: Atlas Integration : Rename Atlas Configurations


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44878/#review123865
---


Ship it!




Ship It!

- Nate Cole


On March 15, 2016, 8:57 p.m., Tom Beerbower wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44878/
> ---
> 
> (Updated March 15, 2016, 8:57 p.m.)
> 
> 
> Review request for Ambari, John Speidel, Nate Cole, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-15431
> https://issues.apache.org/jira/browse/AMBARI-15431
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Atlas configuration name application.properties has been changed to 
> atlas-application.properties to avoid name conflicts with other services. 
> See https://issues.apache.org/jira/browse/ATLAS-392.
> 
> Ambari scripts for Atlas currently use the configuration name 
> application.properties for all stack levels. Stacks which include Atlas > 0.5 
> should use the configuration name atlas-application.properties.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/configuration/atlas-env.xml
>  8500488 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata.py
>  6df47b0 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata_server.py
>  5a39278 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py
>  38c2c9b 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/status_params.py
>  c402721 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon.py
>  b131574 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/params_linux.py
>  115ed4f 
>   
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/params_linux.py
>  ec37243 
>   
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/sqoop.py
>  da67775 
>   
> ambari-server/src/main/resources/stacks/HDP/2.6/services/ATLAS/configuration/atlas-env.xml
>  PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.6/services/ATLAS/metainfo.xml 
> 9514576 
>   ambari-server/src/test/python/stacks/2.3/configs/default.json d5102dc 
>   ambari-server/src/test/python/stacks/2.3/configs/secure.json 2b60be6 
>   ambari-server/src/test/python/stacks/2.6/ATLAS/test_atlas_server.py 
> PRE-CREATION 
>   ambari-server/src/test/python/stacks/2.6/configs/default.json PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/44878/diff/
> 
> 
> Testing
> ---
> 
> Manual test install Atlas (HDP 2.6 stack).  Verify configuration.
> 
> mvn clean test
> 
> all pass
> 
> 
> Thanks,
> 
> Tom Beerbower
> 
>

Re: Review Request 44926: [DRAFT] Auto-retry on failure during RU/EU


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44926/#review124117
---




ambari-server/src/main/java/org/apache/ambari/server/actionmanager/HostRoleCommand.java
 (line 61)


Today, startTime is allowed to be changed to -1 after it already had value. 
For this reason, I added a column called originalStartTime that can only be set 
once.



ambari-server/src/main/java/org/apache/ambari/server/agent/RetryActionMonitor.java
 (line 51)


Switched to guava service :-)



ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
 (line 1123)


Feature will be turned off by default.



ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
 (line 643)


Added column on ambari upgrade.


- Alejandro Fernandez


On March 17, 2016, 11:07 p.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44926/
> ---
> 
> (Updated March 17, 2016, 11:07 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley and Nate Cole.
> 
> 
> Bugs: AMBARI-15446
> https://issues.apache.org/jira/browse/AMBARI-15446
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When a failure occurs during RU/EU and the task transitions to HOLDING_FAILED 
> or HOLDING_TIMEDOUT, want Ambari to automatically retry up to up to x mins. 
> This is useful when a host goes down as Ambari is running a task on it.
> ambari.properties will have 1 new parameter. E.g,. 
> stack-upgrade.max_retry_timeout_mins=15 (by default, will not be present)
> If Ambari Server is restarted, it should be able to recover.
> Today, Action Scheduler increases the attempt_count whenever a task is 
> retried, but it requires resetting the start_time to -1. Because of this, we 
> cannot rely on the start_time property to know when to timeout after several 
> retries.
> 
> For the implementation, will add another thread to Ambari that will monitor 
> failed tasks only during active RU/EU and change the status back to PENDING 
> so that Action Scheduler can reschedule it.
> Luckily, any tasks in HOLDING_TIMEDOUT and HOLDING_FAILED states are 
> blocking, so no other stages are allowed to proceed.
> In order to know when a task was first started, will add a new property to 
> host_role_command table called original_start_time.
> 
> For the agents, we need to ensure that they always write out a response. On 
> the first heartbeat, it should send the status of its last command so we know 
> it failed and Ambari can retry.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionDBAccessorImpl.java
>  429f573 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/HostRoleCommand.java
>  2764b3f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java
>  3a80803 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/HeartbeatProcessor.java
>  a1a686a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/RetryActionMonitor.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
>  9404506 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostRoleCommandDAO.java
>  f5b1cb4 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/HostRoleCommandEntity.java
>  19f0602 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/HostRequest.java
>  9eb514a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/LogicalRequest.java
>  82edbcf 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
>  a803f73 
>   ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 9b4810c 
>   ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql cc3d197 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 07c786d 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql 
> ab6dc93 
>   ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 8e91fde 
>   ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 440ca44 
> 
> Diff: https://reviews.apache.org/r/44926/diff/
> 
> 
> Testing
> ---
> 
> Verified on a live cluster.
> 
> TODO: Still need to make more changes to the implementation, add the config, 
> switch to gauva service, add a column, and add unit tests.
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Re: Review Request 44688: PXF should reach out to active namenode to get the delegation token on HDFS HA clusters



> On March 14, 2016, 5:10 p.m., Jayush Luniya wrote:
> > ambari-server/src/main/resources/common-services/PXF/3.0.0/package/alerts/api_status.py,
> >  line 140
> > 
> >
> > Check for null (i.e. no active namenode)
> 
> bhuvnesh chaudhary wrote:
> Thanks for review Jayush,
> 
> I was testing the behavior when the Namenode is down to essentially test 
> for no active namenode, and 
> https://github.com/apache/ambari/blob/trunk/ambari-common/src/main/python/resource_management/libraries/functions/namenode_ha_utils.py#L135-L143
>  handles if the active_namenode identified is an empty list, and returns 
> UnknownConfiguration(..) if none is found which handles the null / empty case 
> ? So, thought not to handle it here again.
> 
> On the other hand, the response from get_active_namenode returned is 
> 'Configuration parameter 'fs_root' was not found in configurations 
> dictionary!' when no active namenode is found (i turned down HDFS), which is 
> kind of not explanatory, do you think we should make this as an exception in 
> a separate jira ?
> 
> bhuvnesh chaudhary wrote:
> Jayush, Let me know if there is any concern if i drop the issue based on 
> the response provided up.
> 
> bhuvnesh chaudhary wrote:
> Dropping the issue and will merge it. Will revisit in case needed.

@Bhuvnesh,
Sorry have been heads down off late. Go ahead and create a separate JIRA if you 
think the usage needs to be updated.


- Jayush


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44688/#review123456
---


On March 11, 2016, 1:26 a.m., bhuvnesh chaudhary wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44688/
> ---
> 
> (Updated March 11, 2016, 1:26 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, jun aoki, Jayush Luniya, 
> Matt, and Oleksandr Diachenko.
> 
> 
> Bugs: AMBARI-15380
> https://issues.apache.org/jira/browse/AMBARI-15380
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> PXF alerts are not working on secured HDFS HA clusters. When the cluster is 
> HA, Active Namenode should be used to get the delegation token for pxf, 
> however currently it goes to localhost.
> Updated the logic to find out the active namenode and use it.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/PXF/3.0.0/package/alerts/api_status.py
>  871da2b 
> 
> Diff: https://reviews.apache.org/r/44688/diff/
> 
> 
> Testing
> ---
> 
> yes. manually.
> 
> 
> Thanks,
> 
> bhuvnesh chaudhary
> 
>

Re: Review Request 44859: Cannot specify ZooKeeper heap size via UI


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44859/
---

(Updated Березень 17, 2016, 1:22 після полудня)


Review request for Ambari, Andrew Onischuk, Dmitro Lisnichenko, and Dmytro Sen.


Bugs: AMBARI-15429
https://issues.apache.org/jira/browse/AMBARI-15429


Repository: ambari


Description
---

zookeeper-env template has the following:
export SERVER_JVMFLAGS=zk_server_heapsize
However, we don't have any heap size property that is exposed via the UI, so 
1GB heapsize is always used.


Diffs
-

  
ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5.2.0/configuration/zookeeper-env.xml
 7ef62bc 
  
ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5.2.0/package/scripts/params_linux.py
 9ff9125 

Diff: https://reviews.apache.org/r/44859/diff/


Testing
---

mvn clean test


Thanks,

Vitalyi Brodetskyi

Re: Review Request 44941: Report AMS / Grafana Per Disk Metrics and other changes

2016-03-19 Thread Sumit Mohanty


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44941/#review123980
---


Ship it!




Ship It!

- Sumit Mohanty


On March 17, 2016, 1:09 a.m., Sid Wagle wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44941/
> ---
> 
> (Updated March 17, 2016, 1:09 a.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Dmytro Sen, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-15448
> https://issues.apache.org/jira/browse/AMBARI-15448
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> We need to collect and expose per-disk metrics from AMS for iostats counters.
> We can sort the disk names alphabetically, and name the metrics like _1, _2, 
> etc.
> 
> Plus from psutils:
> busy_time: (Linux, FreeBSD) time spent doing actual I/Os (in milliseconds)
> read_merged_count (Linux): number of merged reads (see iostat doc)
> write_merged_count (Linux): number of merged writes (see iostats doc)
> 
> _Additional fixes_:
> - Combined_disk_total metric seems to be wrong : it is a sum() of all disk 
> percentages from different disks
> - Missing ATS and JHS metrics due to wrong config item in 
> hadoop-metrics2.properties file
> - Report number of disks
> 
> 
> Diffs
> -
> 
>   
> ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/config_reader.py
>  a053955 
>   
> ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/controller.py
>  c04a61b 
>   
> ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/host_info.py
>  2e8c442 
>   
> ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/metric_collector.py
>  c28fd03 
>   
> ambari-metrics/ambari-metrics-host-monitoring/src/test/python/core/TestHostInfo.py
>  f07b573 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_grafana_util.py
>  e5020a5 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/templates/metric_monitor.ini.j2
>  0b0932a 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/templates/hadoop-metrics2.properties.j2
>  47b504f 
> 
> Diff: https://reviews.apache.org/r/44941/diff/
> 
> 
> Testing
> ---
> 
> Manually verified.
> 
> --
> Ran 12 tests in 0.020s
> 
> OK
> --
> Total run:12
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Sid Wagle
> 
>

Re: Review Request 44958: Increase Ambari Server Perm gen default value

2016-03-19 Thread Daniel Gergely


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44958/
---

(Updated márc. 18, 2016, 8:39 de)


Review request for Ambari, Laszlo Puskas, Oliver Szabo, Robert Levas, Sandor 
Magyari, and Sebastian Toader.


Changes
---

Removing -XX:+CMSPermGenSweepingEnabled, since it is not used from oracle java 
7.


Bugs: AMBARI-15465
https://issues.apache.org/jira/browse/AMBARI-15465


Repository: ambari


Description
---

Increased permGen size to 128m, also added **-XX:+CMSPermGenSweepingEnabled** 
and **-XX:+CMSClassUnloadingEnabled** to allow jvm to keep permGen space clean.


Diffs (updated)
-

  ambari-server/conf/unix/ambari-env.sh e9fdcdd 
  ambari-server/src/main/python/ambari_server_main.py 00e084b 

Diff: https://reviews.apache.org/r/44958/diff/


Testing
---

Manual testing:
[root@c6401 sbin]# ps -ef | grep ambari-server
root 20769 1 27 14:47 pts/000:00:24 
//usr/jdk64/jdk1.8.0_60/bin/java -server -XX:NewRatio=3 -XX:+UseConcMarkSweepGC 
-XX:-UseGCOverheadLimit -XX:CMSInitiatingOccupancyFraction=60 
**-XX:+CMSPermGenSweepingEnabled -XX:+CMSClassUnloadingEnabled** 
-Dsun.zip.disableMemoryMapping=true -Xms512m -Xmx2048m **-XX:MaxPermSize=128m** 
-Djava.security.auth.login.config=//etc/ambari-server/conf/krb5JAASLogin.conf 
-Djava.security.krb5.conf=/etc/krb5.conf 
-Djavax.security.auth.useSubjectCredsOnly=false -Xms512m -Xmx2048m 
-Djava.security.auth.login.config=//etc/ambari-server/conf/krb5JAASLogin.conf 
-Djava.security.krb5.conf=/etc/krb5.conf 
-Djavax.security.auth.useSubjectCredsOnly=false -cp 
//etc/ambari-server/conf:/usr/lib/ambari-server/*:/usr/share/java/postgresql-jdbc.jar
 org.apache.ambari.server.controller.AmbariServer


Thanks,

Daniel Gergely

Re: Review Request 44956: Topology host info is not cleared when a host is removed

2016-03-19 Thread Sebastian Toader


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44956/#review124030
---




ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostResourceProvider.java
 (line 780)


Please ensure that this method runs within a transaction by using 
@Transactional annotation



ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterTopology.java
 (line 175)


Looks like unfinished comment


- Sebastian Toader


On March 17, 2016, 1:59 p.m., Daniel Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44956/
> ---
> 
> (Updated March 17, 2016, 1:59 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Laszlo Puskas, Oliver Szabo, 
> Sandor Magyari, Sumit Mohanty, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-15457
> https://issues.apache.org/jira/browse/AMBARI-15457
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When a host is removed, a record is left in the topology_host_info table. As 
> a result re-adding the same host is not possible, due to invalid topology.
> Host removal should remove the corresponding record from topology_host_info 
> as well.
> 
> A new column with a foreign key to hosts table is added. In addition 
> ClusterTopologyImpl was statefull and hostGroupInfoMap was not updated when a 
> host was removed.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostResourceProvider.java
>  6101ee8 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/TopologyHostInfoDAO.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/TopologyHostInfoEntity.java
>  a4f251a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClustersImpl.java
>  46dd49d 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterTopology.java
>  4e178c0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterTopologyImpl.java
>  003539c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/HostGroupInfo.java
>  4847a11 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/PersistedState.java
>  dbf6735 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/PersistedStateImpl.java
>  b878955 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/TopologyManager.java
>  41f3150 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog222.java
>  8267d5d 
>   ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql dee397f 
>   ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 253b5d7 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 832d1bc 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql 
> 6c8a09d 
>   ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql a6acb74 
>   ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 2957c5c 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerTest.java
>  3e43ef1 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/HostResourceProviderTest.java
>  69c570e 
> 
> Diff: https://reviews.apache.org/r/44956/diff/
> 
> 
> Testing
> ---
> 
> Tests ran without failures. (2016-03-17 13:30)
> 
> 
> Thanks,
> 
> Daniel Gergely
> 
>

Re: Review Request 44835: AMBARI-15420: Refactor resource_management library



> On March 16, 2016, 5:53 p.m., Alejandro Fernandez wrote:
> > ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py,
> >  line 106
> > 
> >
> > This may not necessarily be a rolling upgrade.

fixed


> On March 16, 2016, 5:53 p.m., Alejandro Fernandez wrote:
> > ambari-common/src/main/python/resource_management/libraries/script/script.py,
> >  line 384
> > 
> >
> > May want to ensure this is also a 3-tuple

padded the result to always return 3-element tuple


- Jayush


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44835/#review123892
---


On March 18, 2016, 7:26 p.m., Jayush Luniya wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44835/
> ---
> 
> (Updated March 18, 2016, 7:26 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Juanjo  
> Marron, Nate Cole, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-15420
> https://issues.apache.org/jira/browse/AMBARI-15420
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Refactor resource_management library, to generalize the library and remove 
> HDP-specific hardcodings
> 
> Specifically,
> 1. Make stack-selector, conf-selector tools stack-driven instead of 
> hardcoding hdp-select and conf-select
> 2. Make stack-root stack-driven instead of hardcoding /usr/hdp
> 3. Make copy_tarball mappings stack-driven 
> 4. Make PACKAGE_DIRS mapping in conf_select use stack-root instead of 
> hardcoding the "/usr/hdp"
> 
> In addition, also added a feature in the stack processing engine to support 
> properties values to be defined a external property file (See 
> tarball_map.json, stack_tools.json in patch)
> 
> Three config properties are added
> 1. cluster-env/stack_root
> 2. cluster-env/stack_tools
> 3. cluster-env/tarball_map
> Corresponding helper functions get_stack_root(), get_stack_tool(), 
> get_tarball_map() are added in script.py, which will set the defaults if 
> these config properties are not defined (ambari-server upgrade scenario needs 
> to be addressed to add these config properties on upgrade). These helper 
> functions are used to remove hardcodings in resource_management library.
> 
> 
> Remaining HDP-specific logic in resource_management library
> 
> 1. conf_select::_valid()
> 2. conf_select::get_hadoop_conf_dir()
> 3. list_ambari_managed_repos::repository_names
> 4. version_select_util::get_component_version()
> 5. script::get_stack_version()
> 6. script::should_expose_component_version()
> 7. get_lzo_packages::get_lzo_packages()
> 
> Refactoring this HDP-specific logic would require "Stack Featurization" 
> (AMBARI-13364) to be in place.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/HostCheckReportFileHandler.py 
> ee7db0a 
>   ambari-common/src/main/python/resource_management/core/utils.py 71d0008 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py
>  770595f 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py
>  647b8b6 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/default.py
>  23383dc 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/dynamic_variable_interpretation.py
>  a20b03c 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/get_stack_version.py
>  f2e6567 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/install_windows_msi.py
>  f1cd9cb 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/repo_version_history.py
>  d585dea 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/setup_ranger_plugin.py
>  4d9d8a4 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/setup_ranger_plugin_xml.py
>  2ccc0c6 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/stack_select.py
>  4a8eeb9 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/stack_tools.py
>  PRE-CREATION 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/version_select_util.py
>  95c5cba 
>   
> ambari-common/src/main/python/resource_management/libraries/script/script.py 
> 1b0116e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java
>  4be4049 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ConfigurationDirectory.java
>  7f21aaa 
>   
>

Re: Review Request 44962: file_system get_mount_point_for_dir works incorrect


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44962/#review124066
---


Ship it!




Ship It!

- Alejandro Fernandez


On March 17, 2016, 4:03 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44962/
> ---
> 
> (Updated March 17, 2016, 4:03 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez and Dmytro Sen.
> 
> 
> Bugs: AMBARI-15468
> https://issues.apache.org/jira/browse/AMBARI-15468
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> **Case 1**:
> 
> 
> [root@c6403 ~]# mount
> /dev/mapper/VolGroup-lv_root on / type ext4 (rw)
> /newFs on /newHdfsDir type ext3 (rw,loop=/dev/loop1)
> 
> [root@c6403 ~]# python
> >>> from resource_management import *
> >>> from resource_management.libraries.functions import file_system
> >>> with Environment() as env:
> ...   print file_system.get_mount_point_for_dir('/newHdfsDir/some_dir')
> ...   print file_system.get_mount_point_for_dir('/newHdfsDir')
> ... 
> 2016-03-17 13:00:33,640 - Mount point for directory /newHdfsDir/some_dir 
> is /
> 2016-03-17 13:00:33,640 - Mount point for directory /newHdfsDir is /
> 
> 
> It should be /newHdfsDir, not /
> 
> **Reason**   
> /newHdfsDir and / have the same count of os separators.
> 
> **Case 2**:
> 
> 
> [root@c6403 ~]# mount
> /dev/mapper/VolGroup-lv_root on / type ext4 (rw)
> /newFs on /a/a type ext3 (rw,loop=/dev/loop2)
> /newFs on /a/a1 type ext3 (rw,loop=/dev/loop3)
> 
> [root@c6403 ~]# python
> >>> from resource_management import *
> >>> from resource_management.libraries.functions import file_system
> >>> with Environment() as env:
> >>> from resource_management import *
> >>> from resource_management.libraries.functions import file_system
> >>> with Environment() as env:
> ...   print file_system.get_mount_point_for_dir('/a/a1/some_dir')
> ...   print file_system.get_mount_point_for_dir('/a/a1')
> ...
> 2016-03-17 13:07:38,899 - Mount point for directory /a/a1/some_dir is /a/a
> 2016-03-17 13:07:38,900 - Mount point for directory /a/a1 is /a/a
> 
> 
> It should be /a/a1, not /a/a
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/test/python/resource_management/TestFileSystem.py 4e0eb63 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/file_system.py
>  39b86dd 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
> 6132c2c 
> 
> Diff: https://reviews.apache.org/r/44962/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 44725: After exporting blueprint from ranger enabled cluster ranger.service.https.attrib.keystore.pass is exported



> On March 14, 2016, 12:27 p.m., Di Li wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java,
> >  line 2668
> > 
> >
> > So any properties end with ".pass" will be marked, yes? That check 
> > seems a bit broad to me, at least broader than checking Password or Secret.
> > 
> > Each property (understandably that only when its defintion follows the 
> > rules) should use "property-type" to indicate if it's a password 
> > "PASSWORD". Can we check this instead of 
> > guessing property name patterns? 
> > 
> > We should honor the setting, so that when user needs to mask a 
> > property, he can either ends the property name with .password or .secret 
> > (this is really a hardcoded logic than stack driven) or configure the 
> > property propertly (stack driven, a more preferrable way).

Hello Di,

Property_type is not accessible to the DAO part. It shows only at UI. So I have 
tried to generalise the expression using Property_value rather than 
property_name. Please give your input for that. 
Thanks


- Amruta


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44725/#review123409
---


On March 17, 2016, 6:52 p.m., Amruta Borkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44725/
> ---
> 
> (Updated March 17, 2016, 6:52 p.m.)
> 
> 
> Review request for Ambari, Di Li and Robert Nettleton.
> 
> 
> Bugs: AMBARI-15338
> https://issues.apache.org/jira/browse/AMBARI-15338
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> After exporting blueprint from ranger enabled cluster 
> ranger.service.https.attrib.keystore.pass is exported.
> Which needs to be removed before using the same blueprint to create another 
> cluster
> Error Show when used same blueprint:
> { "status" : 400, "message" : "Blueprint configuration validation failed: 
> Secret references are not allowed in blueprints, replace following properties 
> with real passwords:\n Config:ranger-admin-site 
> Property:ranger.service.https.attrib.keystore.pass\n" }
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
>  f5e7578 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
>  68d5755 
> 
> Diff: https://reviews.apache.org/r/44725/diff/
> 
> 
> Testing
> ---
> 
> Modified test cases to test for if the properties that end with "pass" are 
> getting filtered. Other properties which have 'pass' else where in the name 
> will not get filtered.
> 
> 
> Thanks,
> 
> Amruta Borkar
> 
>

Re: Review Request 44983: Service version display should be based on Version Definition


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44983/
---

(Updated March 18, 2016, 10:43 a.m.)


Review request for Ambari, Alejandro Fernandez and Jonathan Hurley.


Bugs: AMBARI-15473
https://issues.apache.org/jira/browse/AMBARI-15473


Repository: ambari


Description
---

Add {{stack_services}} to the repository_version endpoint, similar to the 
{{services}} attribute that lists the available services.

This will return a list of all stack services as they pertain to the Version 
Definition File.  This is required to show the correct versions that are 
available for a stack based on the repo_version.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RepositoryVersionResourceProvider.java
 c298e0a 
  
ambari-server/src/main/java/org/apache/ambari/server/state/repository/ManifestServiceInfo.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/state/repository/VersionDefinitionXml.java
 93ac767 
  
ambari-server/src/test/java/org/apache/ambari/server/state/repository/VersionDefinitionTest.java
 f2939c7 
  ambari-server/src/test/resources/version_definition_test.xml 69ea581 

Diff: https://reviews.apache.org/r/44983/diff/


Testing (updated)
---

Manual.  Automated:

Tests run: 3963, Failures: 0, Errors: 0, Skipped: 33

[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 35:40.607s
[INFO] Finished at: Fri Mar 18 10:34:17 EDT 2016
[INFO] Final Memory: 37M/770M
[INFO] 


Thanks,

Nate Cole

Re: Review Request 44868: Changed HAWQ directory descriptions


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44868/
---

(Updated March 16, 2016, 10:42 p.m.)


Review request for Ambari, Alejandro Fernandez, bhuvnesh chaudhary, Goutam 
Tadi, jun aoki, Jayush Luniya, Lav Jain, Oleksandr Diachenko, and Sumit Mohanty.


Changes
---

updated description sentences


Bugs: AMBARI-15433
https://issues.apache.org/jira/browse/AMBARI-15433


Repository: ambari


Description
---

Changed HAWQ directory descriptions


Diffs (updated)
-

  
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-site.xml
 d3aa0ed 

Diff: https://reviews.apache.org/r/44868/diff/


Testing
---

manual.


Thanks,

Alexander Denissov

Re: Review Request 44915: https://issues.apache.org/jira/browse/HAWQ-497


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44915/#review123883
---




ambari-web/app/messages.js (line 2586)


This fix is for another story.


- Goutam Tadi


On March 16, 2016, 4:58 p.m., Goutam Tadi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44915/
> ---
> 
> (Updated March 16, 2016, 4:58 p.m.)
> 
> 
> Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, jun aoki, 
> and Lav Jain.
> 
> 
> Bugs: AMBARI-15435
> https://issues.apache.org/jira/browse/AMBARI-15435
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Update configs for setting virtual segments in HAWQ HashDistributed Tables 
> and random distributed tables
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-site.xml
>  d3aa0ed 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/themes/theme.json 
> aa35311 
>   ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 
> 791d5b8 
>   ambari-web/app/messages.js 5addc9b 
> 
> Diff: https://reviews.apache.org/r/44915/diff/
> 
> 
> Testing
> ---
> 
> Yes. Manually.
> 
> 
> Thanks,
> 
> Goutam Tadi
> 
>

Re: Review Request 45054: AMBARI-15443:Make Host bulk command menu item list stack driven instead of a hardcoded list in UI code

2016-03-19 Thread Zhe (Joe) Wang


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45054/#review124333
---


Ship it!




Ship It!

- Zhe (Joe) Wang


On March 18, 2016, 9:46 p.m., Di Li wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45054/
> ---
> 
> (Updated March 18, 2016, 9:46 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jaimin Jetly, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-15443
> https://issues.apache.org/jira/browse/AMBARI-15443
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> The items described here are the ones on the Hosts tab, in the Actions 
> drop-down list where the UI shows entries such as All Hosts. If user mouses 
> over the All Hosts, it shows a sub-list including Hosts and slave components.
> 
> The slave component items are hardcoded in hosts_table_menu_view.js as shown 
> below. This jira is to put this info into each service's metainfo.xml so that 
> the slave component items can be stack driven.
> 
> components: function () {
> var serviceNames = App.Service.find().mapProperty('serviceName');
> var menuItems = [
> O.create(
> { serviceName: 'HDFS', componentName: 'DATANODE', masterComponentName: 
> 'NAMENODE', componentNameFormatted: 
> Em.I18n.t('dashboard.services.hdfs.datanodes') }
> 
> ),
> O.create(
> { serviceName: 'YARN', componentName: 'NODEMANAGER', masterComponentName: 
> 'RESOURCEMANAGER', componentNameFormatted: 
> Em.I18n.t('dashboard.services.yarn.nodeManagers') }
> 
> ),
> O.create(
> { serviceName: 'HBASE', componentName: 'HBASE_REGIONSERVER', 
> masterComponentName: 'HBASE_MASTER', componentNameFormatted: 
> Em.I18n.t('dashboard.services.hbase.regionServers') }
> 
> ),
> O.create(
> { serviceName: 'STORM', componentName: 'SUPERVISOR', masterComponentName: 
> 'SUPERVISOR', componentNameFormatted: 
> Em.I18n.t('dashboard.services.storm.supervisors') }
> 
> )];
> 
> return menuItems.filter(function (item)
> { return serviceNames.contains(item.serviceName); }
> 
> );
> }.property(),
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/StackServiceComponentResponse.java
>  cfd4e7b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackServiceComponentResourceProvider.java
>  70945ba 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ComponentModule.java
>  a122dc6 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/BulkCommandDefinition.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/ComponentInfo.java 
> 770ee5c 
>   ambari-server/src/main/resources/common-services/HAWQ/2.0.0/metainfo.xml 
> e35b7d8 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/metainfo.xml
>  057e126 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml 
> 75d3bea 
>   ambari-server/src/main/resources/common-services/PXF/3.0.0/metainfo.xml 
> afe27ec 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/metainfo.xml 
> 804374a 
>   
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/metainfo.xml 
> 0f71585 
>   ambari-server/src/main/resources/properties.json 01c15f2 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/ComponentModuleTest.java
>  2f84f04 
>   ambari-web/app/mappers/stack_service_mapper.js 8a65055 
>   ambari-web/app/models/stack_service_component.js 26ff1b8 
>   ambari-web/app/views/main/host/hosts_table_menu_view.js e75b643 
>   ambari-web/test/mappers/stack_service_mapper_test.js 4bc36fe 
> 
> Diff: https://reviews.apache.org/r/45054/diff/
> 
> 
> Testing
> ---
> 
> unit test
> patch a trunk cluster with code change, verify the host bulk command list 
> shown for DataNode, HBase Region server, storm supervisor, etc.
> 
> 
> Thanks,
> 
> Di Li
> 
>

Re: Review Request 44887: Capacity Scheduler view is not working when using custom configuration

2016-03-19 Thread DIPAYAN BHOWMICK


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44887/#review123917
---


Ship it!




Ship It!

- DIPAYAN BHOWMICK


On March 16, 2016, 9:10 a.m., Gaurav Nagar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44887/
> ---
> 
> (Updated March 16, 2016, 9:10 a.m.)
> 
> 
> Review request for Ambari, DIPAYAN BHOWMICK, Nitiraj Rathore, Pallav 
> Kulshreshtha, and Rohit Choudhary.
> 
> 
> Bugs: AMBARI-15438
> https://issues.apache.org/jira/browse/AMBARI-15438
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Added a null check when getting configuration.
> 
> 
> Diffs
> -
> 
>   
> contrib/views/utils/src/main/java/org/apache/ambari/view/utils/ambari/RemoteCluster.java
>  abc71ab 
> 
> Diff: https://reviews.apache.org/r/44887/diff/
> 
> 
> Testing
> ---
> 
> Manually tested on local vm
> 
> 
> Thanks,
> 
> Gaurav Nagar
> 
>

Re: Review Request 45030: Replace 0.0.0.0 with hostname in ams-site/timeline.metrics.service.webapp.address

2016-03-19 Thread Dmytro Sen



> On Март 18, 2016, 3:29 п.п., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java,
> >  line 135
> > 
> >
> > Instead of 'special' can we call this something more meaningful; like 
> > `BIND_ALL_IP_ADDRESS`?

Done.


- Dmytro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45030/#review124198
---


On Март 18, 2016, 3:33 п.п., Dmytro Sen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45030/
> ---
> 
> (Updated Март 18, 2016, 3:33 п.п.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Robert Nettleton, and Sid Wagle.
> 
> 
> Bugs: AMBARI-15480
> https://issues.apache.org/jira/browse/AMBARI-15480
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> We need collector hostname in configs. 
> Replace 0.0.0.0 with hostname in 
> ams-site/timeline.metrics.service.webapp.address
> Implement changes for blueprints, stack_advisor, upgrade catalog
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
>  f5e7578 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog222.java
>  84bb9f3 
>   
> ambari-server/src/main/resources/common-services/ACCUMULO/1.6.1.2.2.0/package/scripts/params.py
>  a21035e 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml
>  99e97f7 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py
>  12c0f25 
>   
> ambari-server/src/main/resources/common-services/FLUME/1.4.0.2.0/package/scripts/params.py
>  29f71a7 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py
>  b32aedf 
>   
> ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/package/scripts/params.py
>  d3a0606 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/params_linux.py
>  ab935a1 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/params.py
>  7153c54 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
> ef82bce 
>   
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/hooks/before-START/scripts/params.py
>  469e709 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
>  68d5755 
>   
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog222Test.java
>  acbf055 
>   ambari-server/src/test/python/stacks/2.0.6/configs/default.json b23d4b9 
>   
> ambari-server/src/test/python/stacks/2.0.6/configs/default_ams_embedded.json 
> 688dbe0 
>   
> ambari-server/src/test/python/stacks/2.0.6/configs/default_hive_non_hdfs.json 
> 6d92182 
>   ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py 
> 921ee23 
>   
> ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade-hdfs-secure.json
>  a43f15a 
>   ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade.json 
> 59cd868 
>   ambari-server/src/test/python/stacks/2.2/configs/ranger-admin-upgrade.json 
> 481b6d4 
>   
> ambari-server/src/test/python/stacks/2.2/configs/ranger-usersync-upgrade.json 
> 200ad23 
>   ambari-server/src/test/python/stacks/2.3/configs/ats_1_5.json 3fe1235 
> 
> Diff: https://reviews.apache.org/r/45030/diff/
> 
> 
> Testing
> ---
> 
> Unit tests passed
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Re: Review Request 44916: StackAdvisor needs to set config map for ams-grafana-env to meet non-null expectations



> On March 16, 2016, 5:14 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/StackAdvisorBlueprintProcessor.java,
> >  line 224
> > 
> >
> > How can this ever be null? An empty config type!?
> 
> Robert Levas wrote:
> This is not config properties from a service config type, but updates 
> generated by the stack advisor. Up until now, this hasn't been null, but it 
> is possible based on the data the stack advisor returns.

My point was that you're adding a ton of null checks around this area of the 
code. It seems safer to maybe have the contract return an empty map? That way 
you don't have to constantly wrap this in an if-statement?


- Jonathan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44916/#review123930
---


On March 16, 2016, 4:55 p.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44916/
> ---
> 
> (Updated March 16, 2016, 4:55 p.m.)
> 
> 
> Review request for Ambari, Dmytro Sen, Jonathan Hurley, Nate Cole, and Sid 
> Wagle.
> 
> 
> Bugs: AMBARI-15442
> https://issues.apache.org/jira/browse/AMBARI-15442
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> The StackAdvisor needs to set a config map for the `ams-grafana-env`, even it 
> is empty, to meet non-null expectations in Kerberos and Blueprint logic. 
> 
> To to this, the following line needs to be added to the StackAdvisor code:
> 
> #stacks/HDP/2.0.6/services/stack_advisor.py:473
> ```
> putGrafanaProperty = self.putProperty(configurations, "ams-grafana-env", 
> services)
> ```
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/StackAdvisorBlueprintProcessor.java
>  ea9fffb 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelperImpl.java
>  be6edc9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
>  f5e7578 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
> 8c18421 
>   
> ambari-server/src/test/java/org/apache/ambari/server/api/services/stackadvisor/StackAdvisorBlueprintProcessorTest.java
>  c4e694b 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/KerberosHelperTest.java
>  f6027f3 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
>  68d5755 
>   ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py 
> 9070c0a 
> 
> Diff: https://reviews.apache.org/r/44916/diff/
> 
> 
> Testing
> ---
> 
> Manually tested in cluster Kerbrerized using the Ambari UI.
> 
> 
> Thanks,
> 
> Robert Levas
> 
>

Re: Review Request 44700: Ambari Views : each view should have separate log file for better troubleshooting

2016-03-19 Thread DIPAYAN BHOWMICK


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44700/#review124000
---


Ship it!




Ship It!

- DIPAYAN BHOWMICK


On March 15, 2016, 12:20 p.m., Nitiraj Rathore wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44700/
> ---
> 
> (Updated March 15, 2016, 12:20 p.m.)
> 
> 
> Review request for Ambari, DIPAYAN BHOWMICK, Jaimin Jetly, Rohit Choudhary, 
> and Yusaku Sako.
> 
> 
> Bugs: AMBARI-14084
> https://issues.apache.org/jira/browse/AMBARI-14084
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Earlier : 
> logs of all the views used to go into ambari-server.log which caused problem 
> in debugging views.
> 
> In this Patch:
> ViewRegistry now looks for file
>  view.log4j.properties using the view archive classloader and loads all the
>  log4j properties in it using PropertyConfigurator. Separate log4j
>  configuraiton files added to all core views.
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/java/org/apache/ambari/server/view/ViewRegistry.java 
> e31037e 
>   contrib/views/capacity-scheduler/pom.xml 5569859 
>   contrib/views/capacity-scheduler/src/main/resources/view.log4j.properties 
> PRE-CREATION 
>   contrib/views/files/pom.xml ccea5ab 
>   contrib/views/files/src/main/resources/view.log4j.properties PRE-CREATION 
>   contrib/views/hive/pom.xml 1f150ff 
>   contrib/views/hive/src/main/resources/view.log4j.properties PRE-CREATION 
>   contrib/views/jobs/pom.xml d797ebd 
>   contrib/views/jobs/src/main/resources/view.log4j.properties PRE-CREATION 
>   contrib/views/pig/pom.xml 97160cc 
>   contrib/views/pig/src/main/resources/view.log4j.properties PRE-CREATION 
>   contrib/views/slider/pom.xml 608988f 
>   contrib/views/slider/src/main/resources/view.log4j.properties PRE-CREATION 
>   contrib/views/storm/src/main/resources/view.log4j.properties PRE-CREATION 
>   contrib/views/tez/pom.xml 8e82c5e 
>   contrib/views/tez/src/main/resources/view.log4j.properties PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/44700/diff/
> 
> 
> Testing
> ---
> 
> Manual Testing Done.
> separate log files for each view was properly generated. Also the log level 
> was getting properly set.
> 
> 
> Thanks,
> 
> Nitiraj Rathore
> 
>

Re: Review Request 44724: AMBARI-12906: Alert notifications are created even if credential fields are left empty

2016-03-19 Thread Qin Liu


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44724/
---

(Updated March 20, 2016, 5:50 a.m.)


Review request for Ambari, Di Li and Jonathan Hurley.


Changes
---

I added isEmptyOrSpaces in Validator to check for empty and white spaces.


Bugs: AMBARI-12906
https://issues.apache.org/jira/browse/AMBARI-12906


Repository: ambari


Description
---

User should not be allowed to save changes if the credential fields are left 
empty when creating new Alert Notifications.

Steps to reproduce:
1. Create New Alert Notification
2. Enter Name = test
3. Tick the checkbox for 'Use Authentication'
4. Keep username and password empty
5. Click on Save
6. New alert notifications is created.


Diffs (updated)
-

  
ambari-web/app/controllers/main/alerts/manage_alert_notifications_controller.js 
79e524e 
  ambari-web/app/messages.js 421143a 
  ambari-web/app/templates/main/alerts/create_alert_notification.hbs 0a83f63 
  ambari-web/app/utils/validator.js 6426163 
  
ambari-web/test/controllers/main/alerts/manage_alert_notifications_controller_test.js
 f16bf4d 

Diff: https://reviews.apache.org/r/44724/diff/


Testing
---

FIX:
1. With the fix, user won't be allowd to to save changes if Email is selected 
for alert notification and Use authentication is chosen and the credential 
fields are left empty when creating new Alert Notifications or editing existing 
Alert Notifications.
2. The error message "Username is required" will be shown if the Username is 
empty.
3. The error message "Password is required" will be shown if the password is 
empty. 
The fix here uses a similar approach that was used for the Password 
Confirmation error handling.

TESTS:
The fix has been tested with HDP UI.
manage_alert_notifications_controller_test.js has also been modified to test 
changes.

The following is the result of running ambari-web tests:

24957 tests complete (31 seconds)
145 tests pending


Thanks,

Qin Liu

Re: Review Request 44972: Improve error logging for install errors during blueprint deployments.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44972/
---

(Updated March 17, 2016, 6:11 p.m.)


Review request for Ambari, Di Li and Sumit Mohanty.


Bugs: AMBARI-15412
https://issues.apache.org/jira/browse/AMBARI-15412


Repository: ambari


Description (updated)
---

Improve error logging for install errors during blueprint deployments. 

Currently a severe error during install of a service component gets logged as a 
WARNing

E.g.:

09 Mar 2016 12:11:45,881 WARN [qtp-ambari-agent-146] HeartBeatHandler:603 - 
Operation failed - may be retried. Service component host: KAFKA_BROKER, host: 
hdtest159.svl.ibm.com Action id12-0


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java
 24fea22 

Diff: https://reviews.apache.org/r/44972/diff/


Testing
---

There are no JUnit test cases, but attached the screenshot which shows modified 
output.


File Attachments


Patch
  
https://reviews.apache.org/media/uploaded/files/2016/03/17/2110d517-0cf4-4477-87e7-935c963e04de__AMBARI-15412.patch
Output
  
https://reviews.apache.org/media/uploaded/files/2016/03/17/923d218f-7ca4-4439-b42f-743511936f94__AMBARI-15412_output.png


Thanks,

Amruta Borkar

Re: Review Request 44972: Improve error logging for install errors during blueprint deployments.

2016-03-19 Thread Di Li


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44972/#review124288
---



Hello Amruta,

You listed the branch as 2.2.2. There is no branch-2.2.2 if you look at 
https://git-wip-us.apache.org/repos/asf?p=ambari.git;a=heads. The 2.2 related 
branches are branch-2.2 and branch-2.2.1. The line you changed also does not 
exist in trunk code. 

Could you please reconsider if the change is necessary for Ambari code and 
which branch it should go into?

Thanks.

- Di Li


On March 18, 2016, 6:32 p.m., Amruta Borkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44972/
> ---
> 
> (Updated March 18, 2016, 6:32 p.m.)
> 
> 
> Review request for Ambari, Di Li and Sid Wagle.
> 
> 
> Bugs: AMBARI-15412
> https://issues.apache.org/jira/browse/AMBARI-15412
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Improve error logging for install errors during blueprint deployments. 
> 
> Currently a severe error during install of a service component gets logged as 
> a WARNing
> 
> E.g.:
> 
> 09 Mar 2016 12:11:45,881 WARN [qtp-ambari-agent-146] HeartBeatHandler:603 - 
> Operation failed - may be retried. Service component host: KAFKA_BROKER, 
> host: hdtest159.svl.ibm.com Action id12-0
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java
>  24fea22 
> 
> Diff: https://reviews.apache.org/r/44972/diff/
> 
> 
> Testing
> ---
> 
> There are no JUnit test cases, but attached the screenshot which shows 
> modified output.
> 
> 
> File Attachments
> 
> 
> Patch
>   
> https://reviews.apache.org/media/uploaded/files/2016/03/17/2110d517-0cf4-4477-87e7-935c963e04de__AMBARI-15412.patch
> Output
>   
> https://reviews.apache.org/media/uploaded/files/2016/03/17/923d218f-7ca4-4439-b42f-743511936f94__AMBARI-15412_output.png
> 
> 
> Thanks,
> 
> Amruta Borkar
> 
>

Re: Review Request 44916: StackAdvisor needs to set config map for ams-grafana-env to meet non-null expectations


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44916/
---

(Updated March 17, 2016, 1:48 p.m.)


Review request for Ambari, Dmytro Sen, Jonathan Hurley, Nate Cole, and Sid 
Wagle.


Bugs: AMBARI-15442
https://issues.apache.org/jira/browse/AMBARI-15442


Repository: ambari


Description
---

The StackAdvisor needs to set a config map for the `ams-grafana-env`, even it 
is empty, to meet non-null expectations in Kerberos and Blueprint logic. 

To to this, the following line needs to be added to the StackAdvisor code:

#stacks/HDP/2.0.6/services/stack_advisor.py:473
```
putGrafanaProperty = self.putProperty(configurations, "ams-grafana-env", 
services)
```


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/recommendations/RecommendationResponse.java
 4a66677 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
54d6d6e 
  ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py 59eff4d 

Diff: https://reviews.apache.org/r/44916/diff/


Testing
---

Manually tested in cluster Kerbrerized using the Ambari UI.


Thanks,

Robert Levas

Re: Review Request 44915: https://issues.apache.org/jira/browse/HAWQ-497


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44915/
---

(Updated March 17, 2016, 2:28 p.m.)


Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, jun aoki, 
and Lav Jain.


Bugs: AMBARI-15435
https://issues.apache.org/jira/browse/AMBARI-15435


Repository: ambari


Description
---

Update configs for setting virtual segments in HAWQ HashDistributed Tables and 
random distributed tables


Diffs (updated)
-

  
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-site.xml
 d187017 
  ambari-server/src/main/resources/common-services/HAWQ/2.0.0/themes/theme.json 
aa35311 
  ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 
8e1aff2 
  ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 7a2a37d 

Diff: https://reviews.apache.org/r/44915/diff/


Testing
---

Yes. tested with test case


Thanks,

Goutam Tadi

Re: Review Request 44915: https://issues.apache.org/jira/browse/HAWQ-497


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44915/
---

(Updated March 17, 2016, 12:47 a.m.)


Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, jun aoki, 
and Lav Jain.


Changes
---

tests updated


Bugs: AMBARI-15435
https://issues.apache.org/jira/browse/AMBARI-15435


Repository: ambari


Description
---

Update configs for setting virtual segments in HAWQ HashDistributed Tables and 
random distributed tables


Diffs (updated)
-

  
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-site.xml
 d187017 
  ambari-server/src/main/resources/common-services/HAWQ/2.0.0/themes/theme.json 
aa35311 
  ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 
8e1aff2 
  ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 1af32d1 

Diff: https://reviews.apache.org/r/44915/diff/


Testing (updated)
---

Yes. tested with test case


Thanks,

Goutam Tadi

Re: Review Request 44916: StackAdvisor needs to set config map for ams-grafana-env to meet non-null expectations



> On March 16, 2016, 5:14 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/StackAdvisorBlueprintProcessor.java,
> >  line 224
> > 
> >
> > How can this ever be null? An empty config type!?

This is not config properties from a service config type, but updates generated 
by the stack advisor. Up until now, this hasn't been null, but it is possible 
based on the data the stack advisor returns.


- Robert


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44916/#review123930
---


On March 16, 2016, 4:55 p.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44916/
> ---
> 
> (Updated March 16, 2016, 4:55 p.m.)
> 
> 
> Review request for Ambari, Dmytro Sen, Jonathan Hurley, Nate Cole, and Sid 
> Wagle.
> 
> 
> Bugs: AMBARI-15442
> https://issues.apache.org/jira/browse/AMBARI-15442
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> The StackAdvisor needs to set a config map for the `ams-grafana-env`, even it 
> is empty, to meet non-null expectations in Kerberos and Blueprint logic. 
> 
> To to this, the following line needs to be added to the StackAdvisor code:
> 
> #stacks/HDP/2.0.6/services/stack_advisor.py:473
> ```
> putGrafanaProperty = self.putProperty(configurations, "ams-grafana-env", 
> services)
> ```
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/StackAdvisorBlueprintProcessor.java
>  ea9fffb 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelperImpl.java
>  be6edc9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
>  f5e7578 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
> 8c18421 
>   
> ambari-server/src/test/java/org/apache/ambari/server/api/services/stackadvisor/StackAdvisorBlueprintProcessorTest.java
>  c4e694b 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/KerberosHelperTest.java
>  f6027f3 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
>  68d5755 
>   ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py 
> 9070c0a 
> 
> Diff: https://reviews.apache.org/r/44916/diff/
> 
> 
> Testing
> ---
> 
> Manually tested in cluster Kerbrerized using the Ambari UI.
> 
> 
> Thanks,
> 
> Robert Levas
> 
>

Review Request 45035: Restarting HDFS Before Upgrade Finalizing Does Not Supply the rollingUpgrade Flag


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45035/
---

Review request for Ambari, Alejandro Fernandez, Nate Cole, and Robert Levas.


Bugs: AMBARI-15482
https://issues.apache.org/jira/browse/AMBARI-15482


Repository: ambari


Description
---

During an upgrade, the HDFS NameNode(s) are restarted with the 
{{-rollingUpgrade}} flag. However, it's possible to get to the end of an 
upgrade and decide to "Finalize Later".

This allows the cluster to run in the upgraded state before committing to the 
upgrade. Full cluster control is returned via the Ambari web client.

Administrators can then decide to restart a NameNode. Upon restarting the 
NameNode, it will produce an error that it was not started with the 
{{rollingUpgrade}} flag. 

It seems that as long as an upgrade has not been finalized, the NameNode(s) 
must be started with the {{rollingUpgrade}} to allow them to function properly. 

STR:
- Perform a rolling upgrade from HDP 2.2 to 2.3 (or 2.3 to 2.4); as long as 
there is a major version change.
- Before finalization, say "Finalize Later". All services should be up and 
green.
- Now restart a NameNode


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/agent/ExecutionCommand.java
 9ea541e 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariActionExecutionHelper.java
 4ef215c 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java
 303f3a4 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java
 07061e1 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/UpgradeEntity.java
 3f1a52b 
  ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java 
ed3c772 
  
ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
 1c7ff61 
  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog222.java
 84bb9f3 
  ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql a85202d 
  ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 9b4810c 
  ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql cc3d197 
  ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 07c786d 
  ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql 
ab6dc93 
  ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 8e91fde 
  ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 440ca44 
  
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py
 e4c8c9c 
  
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py
 02905ec 
  
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py
 905802f 
  ambari-web/app/controllers/main/admin/stack_and_upgrade_controller.js 2dceccc 
  ambari-web/app/utils/ajax/ajax.js 29d0715 

Diff: https://reviews.apache.org/r/45035/diff/


Testing
---

Pending...


Thanks,

Jonathan Hurley

Re: Review Request 44923: Alerts: Allow alerts to emit a structured response


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44923/#review123914
---


Fix it, then Ship it!





ambari-server/src/main/java/org/apache/ambari/server/alerts/AgentHeartbeatAlertRunnable.java
 (lines 20 - 21)


No changes in this file; just import moving, so does it need to be in the 
commit?

Also, defaults should be:
```
java
org
com
```



ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AlertDefinitionResourceProvider.java
 (line 560)


StringUtils.equals() is easier to read (and safer)



ambari-server/src/main/java/org/apache/ambari/server/state/alert/AlertDefinition.java
 (line 50)


Our serialized JSON uses _ instead of camelCase (see ignoreHost).

We should add @SerializedName("help_url") here.



ambari-server/src/main/java/org/apache/ambari/server/state/alert/AlertDefinition.java
 (lines 305 - 311)


StringUtils.equals() cuts the lines in half.



ambari-server/src/main/java/org/apache/ambari/server/state/alert/AlertDefinitionFactory.java
 (lines 20 - 33)


Can you adjust your imports so they are:
```
java
org
com
```



ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
 (line 447)


You have the size defined as 512 in the entity, but 255 here.

I'm fine with either size, but we should be consistent.


- Jonathan Hurley


On March 16, 2016, 3:43 p.m., Vitalyi Brodetskyi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44923/
> ---
> 
> (Updated March 16, 2016, 3:43 p.m.)
> 
> 
> Review request for Ambari, Dmytro Sen and Jonathan Hurley.
> 
> 
> Bugs: AMBARI-15445
> https://issues.apache.org/jira/browse/AMBARI-15445
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> For some alerts, having the ability to include a link in the response would 
> be incredibly beneficial.  Having a structured response format would allow 
> alert creators to add links that operators could click on to help give more 
> context around an alert, or point to a remediation step.  Something like:
> 
> {code}
> {
>   "response": "There were more than 5 soft limits reached in the past 10 
> minutes.  Please click the link to see the log file.",
>   "href": "http://some.server/other/stuff;
> }
> {code}
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/alerts/AgentHeartbeatAlertRunnable.java
>  0611e23 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AlertDefinitionResourceProvider.java
>  0f73ec6 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/AlertDefinitionEntity.java
>  9e21bec 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/AlertDefinitionEntity_.java
>  d2d4cf3 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/alert/AlertDefinition.java
>  9fff5f2 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/alert/AlertDefinitionFactory.java
>  3b4f5fc 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
>  a803f73 
>   ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql a85202d 
>   ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 9b4810c 
>   ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql cc3d197 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 07c786d 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql 
> ab6dc93 
>   ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 8e91fde 
>   ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 440ca44 
> 
> Diff: https://reviews.apache.org/r/44923/diff/
> 
> 
> Testing
> ---
> 
> Tests will be added after patch draft review
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>

Re: Review Request 44926: [DRAFT] Auto-retry on failure during RU/EU


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44926/#review124202
---




ambari-server/src/main/java/org/apache/ambari/server/actionmanager/HostRoleCommand.java
 (lines 334 - 340)


Documentation.



ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java
 (line 130)


Get rid of import.



ambari-server/src/main/java/org/apache/ambari/server/agent/RetryActionMonitor.java
 (line 51)


This class is really scoped for stack upgrades. Yet it's name doesn't 
indicate that nor is it in an upgrade package.

Thoughts on changing the name or moving its package?



ambari-server/src/main/java/org/apache/ambari/server/agent/RetryActionMonitor.java
 (line 72)


Comment on why these are skipped (for those of us who don't have upgrade 
blood in our veins)



ambari-server/src/main/java/org/apache/ambari/server/agent/RetryActionMonitor.java
 (line 84)


Consistency in name with `m_`



ambari-server/src/main/java/org/apache/ambari/server/agent/RetryActionMonitor.java
 (line 89)


Consistency in name with `m_`



ambari-server/src/main/java/org/apache/ambari/server/agent/RetryActionMonitor.java
 (line 105)


Expose via Configuration & ambari.properties



ambari-server/src/main/java/org/apache/ambari/server/agent/RetryActionMonitor.java
 (line 119)


Log.info that it's not going to run, and why.



ambari-server/src/main/java/org/apache/ambari/server/agent/RetryActionMonitor.java
 (line 133)


Use {} for log statements, especially debug ones.



ambari-server/src/main/java/org/apache/ambari/server/agent/RetryActionMonitor.java
 (line 178)


Can we fail fast for things like this?

```
if( null == requestId )
  return
```

Makes multiple if/for/if indents easier to read.



ambari-server/src/main/java/org/apache/ambari/server/agent/RetryActionMonitor.java
 (line 180)


Doesn't the contract ensure a non-null collection? You can eliminate this 
if-statement in that case.



ambari-server/src/main/java/org/apache/ambari/server/agent/RetryActionMonitor.java
 (line 207)


Doc.



ambari-server/src/main/java/org/apache/ambari/server/agent/RetryActionMonitor.java
 (line 225)


Doc



ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostRoleCommandDAO.java
 (lines 317 - 320)


Can you use named parameters here and make this a NamedQuery?


- Jonathan Hurley


On March 17, 2016, 7:07 p.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44926/
> ---
> 
> (Updated March 17, 2016, 7:07 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley and Nate Cole.
> 
> 
> Bugs: AMBARI-15446
> https://issues.apache.org/jira/browse/AMBARI-15446
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When a failure occurs during RU/EU and the task transitions to HOLDING_FAILED 
> or HOLDING_TIMEDOUT, want Ambari to automatically retry up to up to x mins. 
> This is useful when a host goes down as Ambari is running a task on it.
> ambari.properties will have 1 new parameter. E.g,. 
> stack-upgrade.max_retry_timeout_mins=15 (by default, will not be present)
> If Ambari Server is restarted, it should be able to recover.
> Today, Action Scheduler increases the attempt_count whenever a task is 
> retried, but it requires resetting the start_time to -1. Because of this, we 
> cannot rely on the start_time property to know when to timeout after several 
> retries.
> 
> For the implementation, will add another thread to Ambari that will monitor 
> failed tasks only during active RU/EU and change the status back to PENDING 
> so that Action Scheduler can reschedule it.
> Luckily, any tasks in HOLDING_TIMEDOUT and HOLDING_FAILED states are 
> blocking, so no other stages are allowed to proceed.
> In order to know when a task was first started, will add a new property to 
> host_role_command table called original_start_time.
> 
> For the agents, we need to ensure that

Re: Review Request 44565: Hive view: Visual Explain does not work when there are multiple statements in the query editor

2016-03-19 Thread DIPAYAN BHOWMICK


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44565/#review124065
---


Ship it!




Ship It!

- DIPAYAN BHOWMICK


On March 17, 2016, 4:07 p.m., Pallav Kulshreshtha wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44565/
> ---
> 
> (Updated March 17, 2016, 4:07 p.m.)
> 
> 
> Review request for Ambari, DIPAYAN BHOWMICK, Srimanth Gunturi, and Yusaku 
> Sako.
> 
> 
> Bugs: AMBARI-15346
> https://issues.apache.org/jira/browse/AMBARI-15346
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Adding the "Explain | Explain Formatted" only for the last query. Make some 
> modification in "updateValue" function of query editor component to handle 
> this change.
> 
> 
> Diffs
> -
> 
>   
> contrib/views/hive/src/main/resources/ui/hive-web/app/components/query-editor.js
>  10c6bb3 
>   contrib/views/hive/src/main/resources/ui/hive-web/app/controllers/index.js 
> a056759 
> 
> Diff: https://reviews.apache.org/r/44565/diff/
> 
> 
> Testing
> ---
> 
> manual testing done. No extra unit test case needed.
> 
> 
> Thanks,
> 
> Pallav Kulshreshtha
> 
>

Re: Review Request 44725: After exporting blueprint from ranger enabled cluster ranger.service.https.attrib.keystore.pass is exported


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44725/
---

(Updated March 17, 2016, 6:52 p.m.)


Review request for Ambari, Di Li and Robert Nettleton.


Changes
---

Currently Secret References are getting filtered based on Property Name, but as 
properties are ending with difference strings (password, secret, pass), the 
filtering expression needs to be modified every time. 

As these password properties have values that follow the same format 
"SECRET:property_name:version:property_name" we can filter Secret References 
based on property value, rather than property name, as this would make it 
generic.


Bugs: AMBARI-15338
https://issues.apache.org/jira/browse/AMBARI-15338


Repository: ambari


Description
---

After exporting blueprint from ranger enabled cluster 
ranger.service.https.attrib.keystore.pass is exported.
Which needs to be removed before using the same blueprint to create another 
cluster
Error Show when used same blueprint:
{ "status" : 400, "message" : "Blueprint configuration validation failed: 
Secret references are not allowed in blueprints, replace following properties 
with real passwords:\n Config:ranger-admin-site 
Property:ranger.service.https.attrib.keystore.pass\n" }


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
 f5e7578 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
 68d5755 

Diff: https://reviews.apache.org/r/44725/diff/


Testing
---

Modified test cases to test for if the properties that end with "pass" are 
getting filtered. Other properties which have 'pass' else where in the name 
will not get filtered.


Thanks,

Amruta Borkar

Re: Review Request 45041: Strange paths in ambari-server setup



> On March 18, 2016, 5:44 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/python/ambari_server/serverConfiguration.py, line 230
> > 
> >
> > This is doing rstrip instead, if the original string contains leading 
> > "/" this needs to be,
> > root = os.environ["ROOT"]
> > root = ("/" + root.lstrip("/")) if root.startswith("/") else root

Why was this dropped?


- Alejandro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45041/#review124230
---


On March 18, 2016, 5:39 p.m., Vitalyi Brodetskyi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45041/
> ---
> 
> (Updated March 18, 2016, 5:39 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Dmitro Lisnichenko, and Dmytro 
> Sen.
> 
> 
> Bugs: AMBARI-15483
> https://issues.apache.org/jira/browse/AMBARI-15483
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> ambari-server setup command shows strange paths with double slashes in front:
> Successfully installed JDK to //usr/jdk64/
> Downloading JCE Policy archive from 
> http://public-repo-1.hortonworks.com/ARTIFACTS/jce_policy-8.zip to 
> //var/lib/ambari-server/resources/jce_policy-8.zip
> Successfully downloaded JCE Policy archive to 
> //var/lib/ambari-server/resources/jce_policy-8.zip
> Installing JCE policy...
> Completing setup...
> Configuring database...
> Enter advanced database configuration y/n ?
> Configuring database...
> Default properties detected. Using built-in database.
> WARNING: Command chown None //etc/ambari-server/conf/password.dat returned 
> exit code //etc/ambari-server/conf/password.dat with message: chown: invalid 
> user: `None'
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/python/ambari_server/serverConfiguration.py 29187bd 
> 
> Diff: https://reviews.apache.org/r/45041/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>

Re: Review Request 44916: StackAdvisor needs to set config map for ams-grafana-env to meet non-null expectations

2016-03-19 Thread Sid Wagle


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44916/#review123895
---



Shouldn't we also have a non-null check in the backend code?

- Sid Wagle


On March 16, 2016, 4:59 p.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44916/
> ---
> 
> (Updated March 16, 2016, 4:59 p.m.)
> 
> 
> Review request for Ambari, Dmytro Sen, Jonathan Hurley, Nate Cole, and Sid 
> Wagle.
> 
> 
> Bugs: AMBARI-15442
> https://issues.apache.org/jira/browse/AMBARI-15442
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> The StackAdvisor needs to set a config map for the `ams-grafana-env`, even it 
> is empty, to meet non-null expectations in Kerberos and Blueprint logic. 
> 
> To to this, the following line needs to be added to the StackAdvisor code:
> 
> #stacks/HDP/2.0.6/services/stack_advisor.py:473
> ```
> putGrafanaProperty = self.putProperty(configurations, "ams-grafana-env", 
> services)
> ```
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
> 8c18421 
> 
> Diff: https://reviews.apache.org/r/44916/diff/
> 
> 
> Testing
> ---
> 
> Manually tested in cluster Kerbrerized using the Ambari UI.
> 
> 
> Thanks,
> 
> Robert Levas
> 
>

Review Request 44969: After upgrading from Ambari 2.1.2 to 2.2.1, HDP from 2.3.0 to 2.3.4 , user is getting 'Consistency Check Failed' error while trying to save configuration in Ranger, Hive and HBas

2016-03-19 Thread Dmitro Lisnichenko


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44969/
---

Review request for Ambari, Jaimin Jetly and Sebastian Toader.


Bugs: AMBARI-15469
https://issues.apache.org/jira/browse/AMBARI-15469


Repository: ambari


Description
---

PROBLEM:After upgrading from Ambari 2.1.2 to 2.2.1, HDP from 2.3.0 to 2.3.4 , 
user is getting 'Consistency Check Failed' error while trying to save 
configuration in Ranger, Hive and HBase

Please find the attached screenshot, and ambari-server log for more information.

{code}
Error occured in stack advisor.
Error details: list.remove(x): x not in list
14 Mar 2016 14:31:26,241  INFO [qtp-ambari-client-23360] StackAdvisorRunner:71 
- advisor script stderr: Traceback (most recent call last):
File "/var/lib/ambari-server/resources/scripts/stack_advisor.py", line 158, in 

main(sys.argv)
File "/var/lib/ambari-server/resources/scripts/stack_advisor.py", line 109, in 
main
result = stackAdvisor.recommendConfigurations(services, hosts)
File "/var/lib/ambari-server/resources/scripts/../stacks/stack_advisor.py", 
line 569, in recommendConfigurations
calculation(configurations, clusterSummary, services, hosts)
File 
"/var/lib/ambari-server/resources/scripts/./../stacks/HDP/2.3/services/stack_advisor.py",
 line 239, in recommendHBASEConfigurations
super(HDP23StackAdvisor, self).recommendHBASEConfigurations(configurations, 
clusterData, services, hosts)
File 
"/var/lib/ambari-server/resources/scripts/./../stacks/HDP/2.2/services/stack_advisor.py",
 line 665, in recommendHBASEConfigurations
hbaseCoProcessorConfigs['hbase.coprocessor.master.classes'].remove('org.apache.hadoop.hbase.security.access.AccessController')
ValueError: list.remove(x): x not in list
{code}

Looks like property 'hbase.coprocessor.master.classes' is empty. The workaround 
is by putting value org.apache.hadoop.hbase.security.access.AccessController 
there.


Diffs
-

  ambari-server/src/main/resources/stacks/HDP/2.2/services/stack_advisor.py 
5e23b8a 

Diff: https://reviews.apache.org/r/44969/diff/


Testing
---

mvn clean test


Thanks,

Dmitro Lisnichenko

Re: Review Request 44987: Editing tmp dir from single to multiple dir doesn't trigger the backend change


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44987/#review124115
---




ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/common.py
 (line 60)


maybe pass the mode into create_dir_as_hawq_user (default None) so it can 
be used with Directory(mode=xyz) resource and not via chmod ? Then there will 
be no need for this function at all.



ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/common.py
 (line 66)


I wonder if Ambari already supports creating a list of directories from a 
string with comma-separated values via Directory resource ?



ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/hawqsegment.py
 (line 45)


plural, so perhaps "..they should be.." instead of "..it should be.."


- Alexander Denissov


On March 17, 2016, 10:37 p.m., jun aoki wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44987/
> ---
> 
> (Updated March 17, 2016, 10:37 p.m.)
> 
> 
> Review request for Ambari.
> 
> 
> Bugs: AMBARI-15475
> https://issues.apache.org/jira/browse/AMBARI-15475
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Modified "HAWQ Segment Temp Directory” from one [/d1/tmp/hawqsegment] to many 
> [/d1/tmp/hawqsegment,/d2/tmp/hawqsegment,/d3/tmp/hawqsegment,/d4/tmp/hawqsegment,/d5/tmp/hawqsegment]
> and the change is not propagated to segments and the folders are not created.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/common.py
>  d62332d 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/hawqsegment.py
>  0a597b6 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/master_helper.py
>  9482b35 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/params.py
>  1bdc5aa 
>   ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqmaster.py 14b5a5f 
>   ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqsegment.py 6ca83b7 
>   ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqstandby.py eaba496 
> 
> Diff: https://reviews.apache.org/r/44987/diff/
> 
> 
> Testing
> ---
> 
> manually tested.
> 
> test_configure_default (test_hawqsegment.TestHawqSegment) ... ok
> test_install_default (test_hawqsegment.TestHawqSegment) ... ok
> test_start_default (test_hawqsegment.TestHawqSegment) ... ok
> test_stop_default (test_hawqsegment.TestHawqSegment) ... ok
> test_hawq_master_critical 
> (test_alert_component_status.TestAlertComponentStatus) ... ok
> test_hawq_master_ok (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_hawq_segment_critical 
> (test_alert_component_status.TestAlertComponentStatus) ... ok
> test_hawq_segment_ok (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_hawq_standby_critical 
> (test_alert_component_status.TestAlertComponentStatus) ... ok
> test_hawq_standby_ok (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_missing_configs (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_configure_default (test_hawqstandby.TestHawqStandby) ... ok
> test_install_default (test_hawqstandby.TestHawqStandby) ... ok
> test_start_default (test_hawqstandby.TestHawqStandby) ... 2016-03-17 
> 15:33:42,969 - Standby host c6401.ambari.apache.org initialized
> ok
> test_stop_default (test_hawqstandby.TestHawqStandby) ... ok
> test_configure_default (test_hawqmaster.TestHawqMaster) ... ok
> test_install_default (test_hawqmaster.TestHawqMaster) ... ok
> test_start_default (test_hawqmaster.TestHawqMaster) ... 2016-03-17 
> 15:33:43,025 - Active master c6401.ambari.apache.org initialized
> ok
> test_start_localmaster (test_hawqmaster.TestHawqMaster) ... 2016-03-17 
> 15:33:43,049 - Master c6401.ambari.apache.org started
> ok
> test_stop_default (test_hawqmaster.TestHawqMaster) ... ok
> 
> 
> Thanks,
> 
> jun aoki
> 
>

Re: Review Request 45012: Recommendation and validation logic for HAWQ hdfs-client.xml output.replace-datanode-on-failure property

2016-03-19 Thread jun aoki


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45012/#review124221
---


Fix it, then Ship it!





ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py (line 
694)


do you want to inline this since propertyvalue is used only once.



ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py (line 
979)


Do you want to set (constant) variable for a magic number 4?
```
REPLACE_DATANODE_ON_FAILURE_THREASHOLD = 4 # my poor naming skill...
if numSegments > REPLACE_DATANODE_ON_FAILURE_THREASHOLD and value ... :
  ...
elif numSegments <= REPLACE_DATANODE_ON_FAILURE_THREASHOLD and value ... : 
  ...

```

and you can use it on line 694


- jun aoki


On March 18, 2016, 2:01 a.m., Alexander Denissov wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45012/
> ---
> 
> (Updated March 18, 2016, 2:01 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, bhuvnesh chaudhary, Goutam 
> Tadi, jun aoki, Jayush Luniya, Lav Jain, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-15499
> https://issues.apache.org/jira/browse/AMBARI-15499
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Recommendation and validation logic for HAWQ hdfs-client.xml 
> output.replace-datanode-on-failure property
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hdfs-client.xml
>  c2a756a 
>   ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 
> c0bc4e2 
>   ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 
> 4f0b4b7 
> 
> Diff: https://reviews.apache.org/r/45012/diff/
> 
> 
> Testing
> ---
> 
> manual
> 
> --
> Ran 253 tests in 7.111s
> 
> OK
> --
> Total run:927
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Alexander Denissov
> 
>

Review Request 44923: Alerts: Allow alerts to emit a structured response


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44923/
---

Review request for Ambari, Dmytro Sen and Jonathan Hurley.


Bugs: AMBARI-15445
https://issues.apache.org/jira/browse/AMBARI-15445


Repository: ambari


Description
---

For some alerts, having the ability to include a link in the response would be 
incredibly beneficial.  Having a structured response format would allow alert 
creators to add links that operators could click on to help give more context 
around an alert, or point to a remediation step.  Something like:

{code}
{
  "response": "There were more than 5 soft limits reached in the past 10 
minutes.  Please click the link to see the log file.",
  "href": "http://some.server/other/stuff;
}
{code}


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/alerts/AgentHeartbeatAlertRunnable.java
 0611e23 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AlertDefinitionResourceProvider.java
 0f73ec6 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/AlertDefinitionEntity.java
 9e21bec 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/AlertDefinitionEntity_.java
 d2d4cf3 
  
ambari-server/src/main/java/org/apache/ambari/server/state/alert/AlertDefinition.java
 9fff5f2 
  
ambari-server/src/main/java/org/apache/ambari/server/state/alert/AlertDefinitionFactory.java
 3b4f5fc 
  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
 a803f73 
  ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql a85202d 
  ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 9b4810c 
  ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql cc3d197 
  ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 07c786d 
  ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql 
ab6dc93 
  ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 8e91fde 
  ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 440ca44 

Diff: https://reviews.apache.org/r/44923/diff/


Testing
---

Tests will be added after patch draft review


Thanks,

Vitalyi Brodetskyi

Re: Review Request 44953: Download client configurations does not work