Re: Review Request 51165: HSI is not started even when "Enable Interactive Query" is set to Yes on installer wizard

2016-08-16 Thread Yusaku Sako 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51165/#review145952
---


Ship it!




Ship It!

- Yusaku Sako


On Aug. 17, 2016, 1:19 a.m., Jaimin Jetly wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51165/
> ---
> 
> (Updated Aug. 17, 2016, 1:19 a.m.)
> 
> 
> Review request for Ambari, Aleksandr Kovalenko, Zhe (Joe) Wang, Vivek 
> Ratnavel Subramanian, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-18173
> https://issues.apache.org/jira/browse/AMBARI-18173
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> HSI is not started even when "Enable Interactive Query" is set to Yes on 
> installer wizard
> 
> 
> Diffs
> -
> 
>   ambari-web/app/controllers/wizard/step7/assign_master_controller.js 1d21b62 
>   ambari-web/app/routes/installer.js 5b00aa2 
> 
> Diff: https://reviews.apache.org/r/51165/diff/
> 
> 
> Testing
> ---
> 
> Tested manually that patch fixes the issue:
> 
>   29243 tests complete (31 seconds)
>   154 tests pending
> 
> 
> Thanks,
> 
> Jaimin Jetly
> 
>

Re: Review Request 51121: Oozie Hive actions fail when Atlas is installed since Atlas Hive Hooks need to be copied to Oozie Share Lib in HDFS

2016-08-16 Thread Alejandro Fernandez 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51121/#review145947
---




ambari-common/src/main/python/resource_management/libraries/providers/hdfs_resource.py
 (line 337)


"not self.main_resource.resource.recursive_chown and (...)" is new


- Alejandro Fernandez


On Aug. 17, 2016, 1 a.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51121/
> ---
> 
> (Updated Aug. 17, 2016, 1 a.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Sumit Mohanty, and Venkat 
> Ranganathan.
> 
> 
> Bugs: AMBARI-18151
> https://issues.apache.org/jira/browse/AMBARI-18151
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Whenever Oozie submits Hive actions, they fail and the application logs show 
> "Internal Error: 
> java.lang.ClassNotFoundException(org.apache.atlas.hive.hook.HiveHook"
> 
> This is because whenever Atlas is installed, the Hive Hook (which is 
> installed via the Oozie RPM too, in /usr/$stack/$version/atlas/hook/hive/) 
> needs to be copied to the Oozie Share lib in HDFS (e.g., 
> hdfs:///user/oozie/share/lib/lib_$timestamp/hive)
> 
> Further, need to save atlas-application.properties used for Hive to 
> hdfs:///user/oozie/share/lib/lib_$timestamp/hive/
> 
> Lastly, remove the atlas.rest.address property from hive-site.xml
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/constants.py
>  c5cbb3f 
>   
> ambari-common/src/main/python/resource_management/libraries/providers/hdfs_resource.py
>  7abdf5c 
>   
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie.py
>  dd3a692 
>   
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_service.py
>  e330cb5 
>   
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/params_linux.py
>  db048bb 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_features.json
>  ed995b2 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-site.xml
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/51121/diff/
> 
> 
> Testing
> ---
> 
> Verified on a live cluster with HDP 2.5 and Oozie, Hive, Atlas
> 
> Python unit tests passed,
> 
> OK
> --
> Total run:1034
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Re: Review Request 51121: Oozie Hive actions fail when Atlas is installed since Atlas Hive Hooks need to be copied to Oozie Share Lib in HDFS

2016-08-16 Thread Alejandro Fernandez 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51121/#review145946
---




ambari-common/src/main/python/resource_management/libraries/providers/hdfs_resource.py
 (line 337)


Andrew, I had to modify this since recursive_chown show=True should still 
execute the rest of this function to forcibly set the owner.



ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie.py
 (line 404)


In order for this to persist the user and owner, I had to modify 
hdfs_resource.py


- Alejandro Fernandez


On Aug. 17, 2016, 1 a.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51121/
> ---
> 
> (Updated Aug. 17, 2016, 1 a.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Sumit Mohanty, and Venkat 
> Ranganathan.
> 
> 
> Bugs: AMBARI-18151
> https://issues.apache.org/jira/browse/AMBARI-18151
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Whenever Oozie submits Hive actions, they fail and the application logs show 
> "Internal Error: 
> java.lang.ClassNotFoundException(org.apache.atlas.hive.hook.HiveHook"
> 
> This is because whenever Atlas is installed, the Hive Hook (which is 
> installed via the Oozie RPM too, in /usr/$stack/$version/atlas/hook/hive/) 
> needs to be copied to the Oozie Share lib in HDFS (e.g., 
> hdfs:///user/oozie/share/lib/lib_$timestamp/hive)
> 
> Further, need to save atlas-application.properties used for Hive to 
> hdfs:///user/oozie/share/lib/lib_$timestamp/hive/
> 
> Lastly, remove the atlas.rest.address property from hive-site.xml
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/constants.py
>  c5cbb3f 
>   
> ambari-common/src/main/python/resource_management/libraries/providers/hdfs_resource.py
>  7abdf5c 
>   
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie.py
>  dd3a692 
>   
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_service.py
>  e330cb5 
>   
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/params_linux.py
>  db048bb 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_features.json
>  ed995b2 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-site.xml
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/51121/diff/
> 
> 
> Testing
> ---
> 
> Verified on a live cluster with HDP 2.5 and Oozie, Hive, Atlas
> 
> Python unit tests passed,
> 
> OK
> --
> Total run:1034
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Re: Review Request 51154: AMBARI-18169: Falcon log4j configuration should be configurable via Ambari

2016-08-16 Thread Sumit Mohanty 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51154/#review145917
---



Can you add it to 
https://github.com/apache/ambari/blob/trunk/ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/metainfo.xml#L120-L127

- Sumit Mohanty


On Aug. 16, 2016, 10 p.m., Venkat Ranganathan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51154/
> ---
> 
> (Updated Aug. 16, 2016, 10 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-18169
> https://issues.apache.org/jira/browse/AMBARI-18169
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> We should have falcon log4j.xml configurable via ambari
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/configuration/falcon-log4j.xml
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/51154/diff/
> 
> 
> Testing
> ---
> 
> Ran all unit tests and passed
> 
> 
> --
> Ran 267 tests in 6.637s
> 
> OK
> --
> Total run:1120
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Venkat Ranganathan
> 
>

Re: Review Request 51144: AMBARI-18168: Create Atlas log4j changes for failed notifications

2016-08-16 Thread Nahappan Somasundaram 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51144/
---

(Updated Aug. 16, 2016, 12:36 p.m.)


Review request for Ambari, Alejandro Fernandez and Sumit Mohanty.


Bugs: AMBARI-18168
https://issues.apache.org/jira/browse/AMBARI-18168


Repository: ambari


Description
---

AMBARI-18168: Create Atlas log4j changes for failed notifications


Diffs
-

  
ambari-server/src/main/resources/common-services/ATLAS/0.7.0.2.5/configuration/atlas-log4j.xml
 PRE-CREATION 

Diff: https://reviews.apache.org/r/51144/diff/


Testing (updated)
---

Copied 0.1.0.2.3/configuration/atlas-log4j.xml and made the changes specified 
in https://issues.apache.org/jira/browse/ATLAS-


Thanks,

Nahappan Somasundaram

Re: Review Request 51123: AMBARI-18152: 'ambari-server --help' not working

2016-08-16 Thread Nahappan Somasundaram 


> On Aug. 16, 2016, 10:18 a.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/python/ambari_server/serverConfiguration.py, line 236
> > 
> >
> > remove the parentheses aorund Exception

That's the coding style that's used in that file, so I didn't want to change 
that. We can file a "fix coding style" bug and clean up the file.


- Nahappan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51123/#review145880
---


On Aug. 15, 2016, 7:29 p.m., Nahappan Somasundaram wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51123/
> ---
> 
> (Updated Aug. 15, 2016, 7:29 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-18152
> https://issues.apache.org/jira/browse/AMBARI-18152
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> ** Issue: **
> AMBARI-15300 accesses *properties* in ServerConfigDefaults::__init__() in 
> serverConfiguration.py even if get_ambari_properties() fails, resulting in an 
> exception, since *properties* is *-1* on failure.
> 
> File "/usr/lib/python2.6/site-packages/ambari_server/serverConfiguration.py", 
> line 349, in __init__
> self.PID_DIR = properties.get_property(PID_DIR_PROPERTY)
> AttributeError: 'int' object has no attribute 'get_property'
> 
> 
> ** Fix: **
> In get_ambari_properties():
> 1. Check if ROOT is available and then use it since this was the primary 
> issue (key not found).
> 2. Read the properties file in its own block of code so that we still have 
> access to the properties even if we are not able to replace $ROOT
> 3. In case of failure reading the properties file, use default properties in 
> the caller.
> 
> 
> Diffs
> -
> 
>   ambari-server/sbin/ambari-server 046ff4b3d40cdfde0d5aee52040d6996aefba08b 
>   ambari-server/src/main/python/ambari_server/serverConfiguration.py 
> 07159c3613fe5acdf8c3a65190a82b4287f2824b 
> 
> Diff: https://reviews.apache.org/r/51123/diff/
> 
> 
> Testing
> ---
> 
> ** 1. mvn test -DskipSurefireTests **
> 
> --
> Ran 267 tests in 6.698s
> 
> OK
> --
> Total run:1120
> Total errors:0
> Total failures:0
> OK
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 1:09.411s
> [INFO] Finished at: Mon Aug 15 17:48:50 PDT 2016
> [INFO] Final Memory: 56M/835M
> [INFO] 
> 
> 
> ** 2. Manual testing **
> Deployed a VM with the latest build from **trunk** and replaced  
> *serverConfiguration.py* with the one from the local build. Verified that 
> *ambari-server.py --help* works as expected:
> 
> [root@c6401 /]# ambari-server.py --help
> Usage: ambari-server.py [options] action [stack_id os]
> 
> Options:
>   -h, --helpshow this help message and exit
>   -f INIT_SCRIPT_FILE, --init-script-file=INIT_SCRIPT_FILE
> File with setup script
>   -r DROP_SCRIPT_FILE, --drop-script-file=DROP_SCRIPT_FILE
> File with drop script
>   -u UPGRADE_SCRIPT_FILE, --upgrade-script-file=UPGRADE_SCRIPT_FILE
> File with upgrade script
>   -t UPGRADE_STACK_SCRIPT_FILE, 
> --upgrade-stack-script-file=UPGRADE_STACK_SCRIPT_FILE
> File with stack upgrade script
>   -j JAVA_HOME, --java-home=JAVA_HOME
> Use specified java_home.  Must be valid on all hosts
>   -v, --verbose Print verbose status messages
>   -s, --silent  Silently accepts default prompt values
>   -g, --debug   Start ambari-server in debug mode
>   -y, --suspend-start   Freeze ambari-server Java process at startup in debug
> mode
>   --all LDAP sync all option.  Synchronize all LDAP users and
> groups.
>   --existingLDAP sync existing option.  Synchronize existing
> Ambari users and groups only.
>   --users=LDAP_SYNC_USERS
> LDAP sync users option. Specifies the path to a CSV
> file of user names to be synchronized.
>   --groups=LDAP_SYNC_GROUPS
> LDAP sync groups option.  Specifies the path to a CSV
> file of group names to be synchronized.
>   --database=DBMS   Database to use
>

Re: Review Request 51121: [PREVIEW] Oozie Hive actions fail when Atlas is installed since Atlas Hive Hooks need to be copied to Oozie Share Lib in HDFS

2016-08-16 Thread Venkat Ranganathan 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51121/#review145866
---




ambari-server/src/main/resources/stacks/HDP/2.5/services/HDFS/configuration/hive-site.xml
 (line 24)


Why do we need this file?


- Venkat Ranganathan


On Aug. 15, 2016, 5:40 p.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51121/
> ---
> 
> (Updated Aug. 15, 2016, 5:40 p.m.)
> 
> 
> Review request for Ambari, Sumit Mohanty and Venkat Ranganathan.
> 
> 
> Bugs: AMBARI-18151
> https://issues.apache.org/jira/browse/AMBARI-18151
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Whenever Oozie submits Hive actions, they fail and the application logs show 
> "Internal Error: 
> java.lang.ClassNotFoundException(org.apache.atlas.hive.hook.HiveHook"
> 
> This is because whenever Atlas is installed, the Hive Hook (which is 
> installed via the Oozie RPM too, in /usr/$stack/$version/atlas/hook/hive/) 
> needs to be copied to the Oozie Share lib in HDFS (e.g., 
> hdfs:///user/oozie/share/lib/lib_$timestamp/hive)
> 
> Further, need to save atlas-application.properties used for Hive in 
> /etc/oozie/conf/action-conf/hive
> 
> Lastly, remove the atlas.rest.address property from hive-site.xml
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie.py
>  dd3a692 
>   
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_service.py
>  e330cb5 
>   
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/params_linux.py
>  db048bb 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HDFS/configuration/hive-site.xml
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/51121/diff/
> 
> 
> Testing
> ---
> 
> Verified on a live cluster with HDP 2.5 and Oozie, Hive, Atlas
> 
> Python unit tests passed,
> 
> OK
> --
> Total run:1034
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Re: Review Request 50980: JBehave & Docker based integration test framework for Log Search

2016-08-16 Thread Miklos Gergely 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50980/#review145851
---


Ship it!




Ship It!

- Miklos Gergely


On Aug. 16, 2016, 9:26 a.m., Oliver Szabo wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50980/
> ---
> 
> (Updated Aug. 16, 2016, 9:26 a.m.)
> 
> 
> Review request for Ambari, Dharmesh Makwana, Miklos Gergely, Robert 
> Nettleton, Sandor Magyari, Sumit Mohanty, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-18076
> https://issues.apache.org/jira/browse/AMBARI-18076
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Docker based test framework for Log Search with JBehave.
> - define stories instead of writing code (for checking component logs are 
> parsed)
> - later add new steps (like using web driver to check to UI too)
> - useful to check integration tests after development to avoid regression 
> issues
> run it from ambari-logsearch folder: mvn clean integration-test 
> -Dit.skip=false
> 
> 
> Diffs
> -
> 
>   ambari-logsearch/README.md 23790cd 
>   ambari-logsearch/ambari-logsearch-it/pom.xml PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-it/src/test/java/org/apache/ambari/logsearch/domain/StoryDataRegistry.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-it/src/test/java/org/apache/ambari/logsearch/steps/LogSearchDockerSteps.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-it/src/test/java/org/apache/ambari/logsearch/steps/SolrSteps.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-it/src/test/java/org/apache/ambari/logsearch/story/LogSearchStory.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-it/src/test/java/org/apache/ambari/logsearch/story/LogfeederParsingStory.java
>  PRE-CREATION 
>   ambari-logsearch/ambari-logsearch-it/src/test/resources/log4j.properties 
> PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-it/src/test/resources/org/apache/ambari/logsearch/story/logfeeder_parsing_story.story
>  PRE-CREATION 
>   ambari-logsearch/pom.xml 0445c39 
> 
> Diff: https://reviews.apache.org/r/50980/diff/
> 
> 
> Testing
> ---
> 
> done.
> 
> 
> Thanks,
> 
> Oliver Szabo
> 
>

Re: Review Request 50980: JBehave & Docker based integration test framework for Log Search

2016-08-16 Thread Oliver Szabo 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50980/
---

(Updated Aug. 16, 2016, 9:26 a.m.)


Review request for Ambari, Dharmesh Makwana, Miklos Gergely, Robert Nettleton, 
Sandor Magyari, Sumit Mohanty, and Sebastian Toader.


Changes
---

add run command


Bugs: AMBARI-18076
https://issues.apache.org/jira/browse/AMBARI-18076


Repository: ambari


Description (updated)
---

Docker based test framework for Log Search with JBehave.
- define stories instead of writing code (for checking component logs are 
parsed)
- later add new steps (like using web driver to check to UI too)
- useful to check integration tests after development to avoid regression issues
run it from ambari-logsearch folder: mvn clean integration-test -Dit.skip=false


Diffs
-

  ambari-logsearch/README.md 23790cd 
  ambari-logsearch/ambari-logsearch-it/pom.xml PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-it/src/test/java/org/apache/ambari/logsearch/domain/StoryDataRegistry.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-it/src/test/java/org/apache/ambari/logsearch/steps/LogSearchDockerSteps.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-it/src/test/java/org/apache/ambari/logsearch/steps/SolrSteps.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-it/src/test/java/org/apache/ambari/logsearch/story/LogSearchStory.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-it/src/test/java/org/apache/ambari/logsearch/story/LogfeederParsingStory.java
 PRE-CREATION 
  ambari-logsearch/ambari-logsearch-it/src/test/resources/log4j.properties 
PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-it/src/test/resources/org/apache/ambari/logsearch/story/logfeeder_parsing_story.story
 PRE-CREATION 
  ambari-logsearch/pom.xml 0445c39 

Diff: https://reviews.apache.org/r/50980/diff/


Testing
---

done.


Thanks,

Oliver Szabo