Re: Review Request 49977: Zeppelin service: Remove sample notebook downloading shell script

2016-07-12 Thread Sumit Mohanty 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49977/#review142012
---


Ship it!




Ship It!

- Sumit Mohanty


On July 13, 2016, 5:38 a.m., Renjith Kamath wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49977/
> ---
> 
> (Updated July 13, 2016, 5:38 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, DIPAYAN BHOWMICK, Gaurav 
> Nagar, Pallav Kulshreshtha, Prabhjyot Singh, Rohit Choudhary, and Sumit 
> Mohanty.
> 
> 
> Bugs: AMBARI-17672
> https://issues.apache.org/jira/browse/AMBARI-17672
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> - fix sample notebook download script failure due to lack of permissions
> - info: sample notebook download is handled in the post install script
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/master.py
>  54256da 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/setup_snapshot.sh
>  47a6ed9 
> 
> Diff: https://reviews.apache.org/r/49977/diff/
> 
> 
> Testing
> ---
> 
> manually tested on centos 6
> 
> 
> Thanks,
> 
> Renjith Kamath
> 
>

Review Request 49977: Zeppelin service: Remove sample notebook downloading shell script

2016-07-12 Thread Renjith Kamath 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49977/
---

Review request for Ambari, Alejandro Fernandez, DIPAYAN BHOWMICK, Gaurav Nagar, 
Pallav Kulshreshtha, Prabhjyot Singh, Rohit Choudhary, and Sumit Mohanty.


Bugs: AMBARI-17672
https://issues.apache.org/jira/browse/AMBARI-17672


Repository: ambari


Description
---

- fix sample notebook download script failure due to lack of permissions
- info: sample notebook download is handled in the post install script


Diffs
-

  
ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/master.py
 54256da 
  
ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/setup_snapshot.sh
 47a6ed9 

Diff: https://reviews.apache.org/r/49977/diff/


Testing
---

manually tested on centos 6


Thanks,

Renjith Kamath

Re: Review Request 49974: Zeppelin service: wrong principal format in zeppelin kerberos.json

2016-07-12 Thread Sumit Mohanty 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49974/#review142001
---


Ship it!




Ship It!

- Sumit Mohanty


On July 13, 2016, 4:05 a.m., Renjith Kamath wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49974/
> ---
> 
> (Updated July 13, 2016, 4:05 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, DIPAYAN BHOWMICK, Gaurav 
> Nagar, Pallav Kulshreshtha, Prabhjyot Singh, Rohit Choudhary, and Sumit 
> Mohanty.
> 
> 
> Bugs: AMBARI-17677
> https://issues.apache.org/jira/browse/AMBARI-17677
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Expected format : 
> ${zeppelin-env/zeppelin_user}-${cluster_name|toLower()}@${realm}
> Current format: zeppelin/${cluster_name}@${realm}
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/kerberos.json
>  cc6ab09 
> 
> Diff: https://reviews.apache.org/r/49974/diff/
> 
> 
> Testing
> ---
> 
> Manually tested on centos
> 
> 
> Thanks,
> 
> Renjith Kamath
> 
>

Review Request 49974: Zeppelin service: wrong principal format in zeppelin kerberos.json

2016-07-12 Thread Renjith Kamath 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49974/
---

Review request for Ambari, Alejandro Fernandez, DIPAYAN BHOWMICK, Gaurav Nagar, 
Pallav Kulshreshtha, Prabhjyot Singh, Rohit Choudhary, and Sumit Mohanty.


Bugs: AMBARI-17677
https://issues.apache.org/jira/browse/AMBARI-17677


Repository: ambari


Description
---

Expected format : 
${zeppelin-env/zeppelin_user}-${cluster_name|toLower()}@${realm}
Current format: zeppelin/${cluster_name}@${realm}


Diffs
-

  
ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/kerberos.json
 cc6ab09 

Diff: https://reviews.apache.org/r/49974/diff/


Testing
---

Manually tested on centos


Thanks,

Renjith Kamath

Re: Review Request 48972: AMBARI-17253 Ambari Alert causes too many wanings in ZooKeeper logs.

2016-07-12 Thread Jonathan Hurley 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48972/#review141993
---


Ship it!




I'm going to +1 this, but I'd still like to know how other commiters feel about 
making the socket commands paramters vs actual parts of the source element.

- Jonathan Hurley


On July 11, 2016, 5:50 p.m., Masahiro Tanaka wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48972/
> ---
> 
> (Updated July 11, 2016, 5:50 p.m.)
> 
> 
> Review request for Ambari, Florian Barca, Jonathan Hurley, and Nate Cole.
> 
> 
> Bugs: AMBARI-17253
> https://issues.apache.org/jira/browse/AMBARI-17253
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> There are too many WARNING in ZooKeeper log.
> ```
> 2016-06-15 21:02:15,405 - WARN  
> [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@357] - caught end of 
> stream exception
> EndOfStreamException: Unable to read additional data from client sessionid 
> 0x0, likely client has closed socket
> at 
> org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
> at 
> org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
> at java.lang.Thread.run(Thread.java:745)
> ```
> 
> It may be because of Ambari Alert. Ambari Alert pings to the zookeeper port 
> to do monitoring.
> We should use 'ruok' to monitor zookeepers.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/alerts/port_alert.py 1918327 
>   ambari-agent/src/test/python/ambari_agent/TestPortAlert.py dffa56c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/alert/PortSource.java
>  d7279de 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
>  7ef12a7 
>   
> ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5/alerts.json 
> 469036a 
> 
> Diff: https://reviews.apache.org/r/48972/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> ```
> +1 overall. Here are the results of testing the latest attachment 
> http://issues.apache.org/jira/secure/attachment/12811835/AMBARI-17253.2.patch
> against trunk revision .
> +1 @author. The patch does not contain any @author tags.
> +1 tests included. The patch appears to include 1 new or modified test files.
> +1 javac. The applied patch does not increase the total number of javac 
> compiler warnings.
> +1 release audit. The applied patch does not increase the total number of 
> release audit warnings.
> +1 core tests. The patch passed unit tests in .
> Test results: 
> https://builds.apache.org/job/Ambari-trunk-test-patch/7427//testReport/
> Console output: 
> https://builds.apache.org/job/Ambari-trunk-test-patch/7427//console
> This message is automatically generated.
> ```
> 
> 
> Thanks,
> 
> Masahiro Tanaka
> 
>

Re: Review Request 48972: AMBARI-17253 Ambari Alert causes too many wanings in ZooKeeper logs.

2016-07-12 Thread Jonathan Hurley 


> On July 11, 2016, 5:51 p.m., Masahiro Tanaka wrote:
> > Should I UpgradeCatalog250.java as we are readying to release Ambari2.4.0?

That depends; there's no Fix Version on this Jira, so I don't know if it's only 
going into trunk or if it's going to into branch-2.4. If you get approval to 
put this in branch 2.4 from the 2.4 release manager, then you can add it to 
UgpradeCatalog240. If this is going only in trunk, then you'll want to create 
it a new UpgradeCatalog250.


- Jonathan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48972/#review141786
---


On July 11, 2016, 5:50 p.m., Masahiro Tanaka wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48972/
> ---
> 
> (Updated July 11, 2016, 5:50 p.m.)
> 
> 
> Review request for Ambari, Florian Barca, Jonathan Hurley, and Nate Cole.
> 
> 
> Bugs: AMBARI-17253
> https://issues.apache.org/jira/browse/AMBARI-17253
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> There are too many WARNING in ZooKeeper log.
> ```
> 2016-06-15 21:02:15,405 - WARN  
> [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@357] - caught end of 
> stream exception
> EndOfStreamException: Unable to read additional data from client sessionid 
> 0x0, likely client has closed socket
> at 
> org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
> at 
> org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
> at java.lang.Thread.run(Thread.java:745)
> ```
> 
> It may be because of Ambari Alert. Ambari Alert pings to the zookeeper port 
> to do monitoring.
> We should use 'ruok' to monitor zookeepers.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/alerts/port_alert.py 1918327 
>   ambari-agent/src/test/python/ambari_agent/TestPortAlert.py dffa56c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/alert/PortSource.java
>  d7279de 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
>  7ef12a7 
>   
> ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5/alerts.json 
> 469036a 
> 
> Diff: https://reviews.apache.org/r/48972/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> ```
> +1 overall. Here are the results of testing the latest attachment 
> http://issues.apache.org/jira/secure/attachment/12811835/AMBARI-17253.2.patch
> against trunk revision .
> +1 @author. The patch does not contain any @author tags.
> +1 tests included. The patch appears to include 1 new or modified test files.
> +1 javac. The applied patch does not increase the total number of javac 
> compiler warnings.
> +1 release audit. The applied patch does not increase the total number of 
> release audit warnings.
> +1 core tests. The patch passed unit tests in .
> Test results: 
> https://builds.apache.org/job/Ambari-trunk-test-patch/7427//testReport/
> Console output: 
> https://builds.apache.org/job/Ambari-trunk-test-patch/7427//console
> This message is automatically generated.
> ```
> 
> 
> Thanks,
> 
> Masahiro Tanaka
> 
>

Re: Review Request 49640: Identify config changes added to Ambari-2.4.0 and mark them to not get added during Ambari upgrade

2016-07-12 Thread Jonathan Hurley 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49640/#review141990
---


Ship it!




Have you tested this in an upgrade scenario to ensure that our Python code 
doesn't make some bad assumptions about properties being present? The last 
thing we want is to have Python errors on missing properties after upgrade.

- Jonathan Hurley


On July 11, 2016, 3:41 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49640/
> ---
> 
> (Updated July 11, 2016, 3:41 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Nate Cole, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17564
> https://issues.apache.org/jira/browse/AMBARI-17564
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Now that we have the mechanism to prevent configs from getting added as part 
> of Ambari upgrade, lets identify the configs that got added in 2.4.0 and mark 
> them as not to be added during Ambari upgrade.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-hbase-env.xml
>  b4eecec 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml
>  871e571 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/storm-site.xml
>  61b4233 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params_linux.py
>  52e9fe5 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/configuration/atlas-env.xml
>  8087204 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py
>  68961c1 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/configuration/hbase-env.xml
>  9811191 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py
>  0602d11 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-site.xml
>  61437d5 
>   
> ambari-server/src/main/resources/common-services/KERBEROS/1.10.3-10/configuration/kerberos-env.xml
>  29c46e9 
>   
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/configuration/oozie-env.xml
>  d0e51eb 
>   
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/params_linux.py
>  c158894 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-tagsync-site.xml
>  a0bd322 
>   
> ambari-server/src/main/resources/stacks/HDP/2.2/services/YARN/configuration/yarn-site.xml
>  fd0cb3e 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/STORM/configuration/storm-site.xml
>  f3bbce8 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/tez-interactive-site.xml
>  8d8af19 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/RANGER/configuration/ranger-tagsync-site.xml
>  6a0991b 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/STORM/configuration/storm-site.xml
>  902fdc9 
>   
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/OOZIE/configuration/oozie-site.xml
>  766fe62 
> 
> Diff: https://reviews.apache.org/r/49640/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Re: Review Request 48734: App timeline Server start fails on enabling HA because namenode is in safemode

2016-07-12 Thread Jonathan Hurley 


> On June 21, 2016, 4:48 p.m., Jonathan Hurley wrote:
> > Ship It!
> 
> Victor Galgo wrote:
> Jonathan can please do the honours of helping to commit this patch?

Has this been committed yet? If so, please close the review.


- Jonathan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48734/#review138935
---


On June 17, 2016, 6:45 p.m., Victor Galgo wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48734/
> ---
> 
> (Updated June 17, 2016, 6:45 p.m.)
> 
> 
> Review request for Ambari, Andriy Babiichuk, Alexandr Antonenko, Andrew 
> Onischuk, Di Li, Dmitro Lisnichenko, Jonathan Hurley, Jayush Luniya, Robert 
> Levas, Sandor Magyari, Sumit Mohanty, Sebastian Toader, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-17182
> https://issues.apache.org/jira/browse/AMBARI-17182
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> On the last step "Start all" on enabling HA below happens:
> 
> Traceback (most recent call last):
> File 
> "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/application_timeline_server.py",
>  line 147, in 
>   ApplicationTimelineServer().execute()
> File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 219, in execute
>   method(env)
> File 
> "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/application_timeline_server.py",
>  line 43, in start
>   self.configure(env) # FOR SECURITY
> File 
> "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/application_timeline_server.py",
>  line 54, in configure
>   yarn(name='apptimelineserver')
> File "/usr/lib/python2.6/site-packages/ambari_commons/os_family_impl.py", 
> line 89, in thunk
>   return fn(*args, **kwargs)
> File 
> "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py",
>  line 276, in yarn
>   mode=0755
> File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", 
> line 154, in __init__
>   self.env.run()
> File 
> "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
> line 160, in run
>   self.run_action(resource, action)
> File 
> "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
> line 124, in run_action
>   provider_action()
> File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py",
>  line 463, in action_create_on_execute
>   self.action_delayed("create")
> File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py",
>  line 460, in action_delayed
>   self.get_hdfs_resource_executor().action_delayed(action_name, self)
> File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py",
>  line 259, in action_delayed
>   self._set_mode(self.target_status)
> File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py",
>  line 366, in _set_mode
>   self.util.run_command(self.main_resource.resource.target, 
> 'SETPERMISSION', method='PUT', permission=self.mode, assertable_result=False)
> File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py",
>  line 195, in run_command
>   raise Fail(err_msg)
>   resource_management.core.exceptions.Fail: Execution of 'curl -sS -L -w 
> '%{http_code}' -X PUT 
> 'http://testvgalgo.org:50070/webhdfs/v1/ats/done?op=SETPERMISSION=hdfs=755''
>  returned status_code=403. 
>   {
> "RemoteException": {
>   "exception": "RetriableException", 
>   "javaClassName": "org.apache.hadoop.ipc.RetriableException", 
>   "message": "org.apache.hadoop.hdfs.server.namenode.SafeModeException: 
> Cannot set permission for /ats/done. Name node is in safe mode.\nThe reported 
> blocks 675 needs additional 16 blocks to reach the threshold 0.9900 of total 
> blocks 697.\nThe number of live datanodes 20 has reached the minimum number 
> 0. Safe mode will be turned off automatically once the thresholds have been 
> reached."
> }
>   }
>   
>   
> This happens because NN is not yet out of safemode at the moment of ats 
> start, because DNs just started.
> 
> To fix this "stop namenodes" has to be triggered before "start all".
> 
> If this is done, on "Start all" it will be ensured that datanodes start prior 
> to NN, and that NN are out of safemode before ATS start.
> 
> 
> Diffs
> -
> 
>   
> ambari-web/app/controllers/main/admin/highAvailability/nameNode/step9_controller.js
>  24677e4 
>   ambari-web/app/messages.js 6465812 
> 
> Diff:

Re: Review Request 49787: AMBARI-17623: Update default values of nimbus.monitor.freq.secs to 10 secs and zookeeper.session.timeout to 30 secs

2016-07-12 Thread Satish Duggana 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49787/
---

(Updated July 13, 2016, 12:09 a.m.)


Review request for Ambari and Apache Ambari.


Bugs: AMBARI-17623
https://issues.apache.org/jira/browse/AMBARI-17623


Repository: ambari


Description
---

Update default values of nimbus.monitor.freq.secs to 10 secs and 
zookeeper.session.timeout to 30 secs


Diffs
-

  
ambari-server/src/main/resources/common-services/STORM/0.9.1/configuration/storm-site.xml
 157fb5c 

Diff: https://reviews.apache.org/r/49787/diff/


Testing
---


Thanks,

Satish Duggana

Re: Review Request 49930: AMBARI-17614 Clean up import * for AMBARI_METRICS services

2016-07-12 Thread Aravindan Vijayan 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49930/#review141981
---


Ship it!




Ship It!

- Aravindan Vijayan


On July 11, 2016, 10:59 p.m., Masahiro Tanaka wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49930/
> ---
> 
> (Updated July 11, 2016, 10:59 p.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Jayush Luniya, and Juanjo  
> Marron.
> 
> 
> Bugs: AMBARI-17614
> https://issues.apache.org/jira/browse/AMBARI-17614
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/status.py
>  and 
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_monitor.py
> uses `from resource_management import *`. It increases code tracking 
> difficulty.
> I think this is related to AMBARI-16101
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_monitor.py
>  7dc66b2 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/status.py
>  fa4e8a4 
> 
> Diff: https://reviews.apache.org/r/49930/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Masahiro Tanaka
> 
>

Re: Review Request 49971: Misc Atlas fixes, remove conf dir from classpath of Falcon and Storm in HDP 2.5, add more security properties to Atlas Hooks, and delete deprecated configs

2016-07-12 Thread Alejandro Fernandez 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49971/#review141973
---




ambari-common/src/main/python/resource_management/libraries/functions/setup_atlas_hook.py
 (line 79)


These properties will now be written out to any Atlas Hook's 
atlas-application.properties.xml file



ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py
 (line 130)


Moved these properties up top so could write out to the 
application.properties file


- Alejandro Fernandez


On July 12, 2016, 11 p.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49971/
> ---
> 
> (Updated July 12, 2016, 11 p.m.)
> 
> 
> Review request for Ambari, Madhan Neethiraj, Nahappan Somasundaram, Sumit 
> Mohanty, and Suma Shivaprasad.
> 
> 
> Bugs: AMBARI-17678
> https://issues.apache.org/jira/browse/AMBARI-17678
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Miscellaneous fixes for Atlas.
> 1. Atlas 0.7.0 in HDP 2.5 no longer needs to add the Atlas conf dir to the 
> classpath of Falcon and Storm.
> This means we need to call check_stack_feature for 
> StackFeature.ATLAS_CONF_DIR_IN_PATH which allows HDP [2.3, 2.5)
> 
> 2. The Atlas Hooks creates the atlas-application.properties.xml file with a 
> subset of the Atlas application properties, which is missing two properties,
> 
> * atlas.kafka.sasl.kerberos.service.name
> * atlas.kafka.security.protocol
> 
> 
> 3. Change the Atlas log level to info. Delete the following properties in 
> Atlas 0.7.0:
> 
> * atlas.kafka.data
> * atlas.graph.index.search.directory
> * atlas.graph.index.search.elasticsearch.client-only
> * atlas.graph.index.search.elasticsearch.local-mode
> * atlas.graph.storage.directory
> * atlas.kafka.entities.group.id
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/setup_atlas_hook.py
>  c7f6204 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/configuration/atlas-log4j.xml
>  372fd31 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py
>  8445214 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.7.0.2.5/configuration/application-properties.xml
>  aa9940e 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/params_linux.py
>  e58d205 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/params_linux.py
>  2afb187 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_features.json
>  7f85ad3 
>   ambari-server/src/test/python/stacks/2.3/ATLAS/test_metadata_server.py 
> 9e1c7f7 
>   ambari-server/src/test/python/stacks/2.3/configs/default.json 080274f 
>   ambari-server/src/test/python/stacks/2.3/configs/secure.json a4e7096 
>   ambari-server/src/test/python/stacks/2.5/ATLAS/test_atlas_server.py b3f2acc 
>   ambari-server/src/test/python/stacks/2.5/configs/default.json cf36728 
> 
> Diff: https://reviews.apache.org/r/49971/diff/
> 
> 
> Testing
> ---
> 
> python unit tests passed
> 
> --
> Total run:1018
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Re: Review Request 49935: AMBARI-17633 yarn.nodemanager.remote-app-log-dir should be added stickybit.

2016-07-12 Thread Masahiro Tanaka 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49935/
---

(Updated July 13, 2016, 6:55 a.m.)


Review request for Ambari, Andrew Onischuk, jun aoki, and Myroslav Papirkovskyy.


Changes
---

Update the patch


Bugs: AMBARI-17633
https://issues.apache.org/jira/browse/AMBARI-17633


Repository: ambari


Description
---

When installing YARN+MapReduce2, I got a WARN message in nodemanager log(in Log 
Search View) like below
```
2016-07-09 06:30:21,865 WARN logaggregation.LogAggregationService 
LogAggregationService.java:230 - Remote Root Log Dir [/app-logs] already exist, 
but with incorrect permissions. Expected: [rwxrwxrwt], Found: [rwxrwxrwx]. The 
cluster may have problems with multiple users. 
```
I think the cause of this WARN is 
[this](https://github.com/apache/ambari/blob/trunk/ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py#L115).
We should add stickybit to yarn.nodemanager.remote-app-log-dir.


Diffs (updated)
-

  
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py
 cf5736e 
  ambari-server/src/test/python/stacks/2.0.6/YARN/test_historyserver.py 4600063 

Diff: https://reviews.apache.org/r/49935/diff/


Testing
---

mvn clean test && manual test

I think Hadoop QA result is unrelated to this patch


Thanks,

Masahiro Tanaka

Re: Review Request 49895: AMBARI-17631: preinstall-check script should use AMBARI-AGENT REST API for the list of agents

2016-07-12 Thread Juanjo Marron 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49895/#review141961
---


Ship it!




Ship It!

- Juanjo  Marron


On July 12, 2016, 4:48 p.m., Di Li wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49895/
> ---
> 
> (Updated July 12, 2016, 4:48 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez and Juanjo  Marron.
> 
> 
> Bugs: AMBARI-17631
> https://issues.apache.org/jira/browse/AMBARI-17631
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> it should use the existing services/AMBARI/components/AMBARI-AGENT rest api 
> for the list of agents
> 
> 
> Diffs
> -
> 
>   contrib/utils/preinstall-check/src/main/python/preinstall_checker.py 
> 7ff7e4c 
> 
> Diff: https://reviews.apache.org/r/49895/diff/
> 
> 
> Testing
> ---
> 
> Run the script against a cluster just Ambari server and Ambari Agent nodes 
> installed.verify the script can get the correct list of ambari agents.
> 
> 
> Thanks,
> 
> Di Li
> 
>

Re: Review Request 49927: Atlas HA fails to come up with error finding ids

2016-07-12 Thread Sumit Mohanty 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49927/#review141959
---


Ship it!




Ship It!

- Sumit Mohanty


On July 11, 2016, 9:42 p.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49927/
> ---
> 
> (Updated July 11, 2016, 9:42 p.m.)
> 
> 
> Review request for Ambari, Madhan Neethiraj, Nahappan Somasundaram, Sumit 
> Mohanty, and Suma Shivaprasad.
> 
> 
> Bugs: AMBARI-17662
> https://issues.apache.org/jira/browse/AMBARI-17662
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Deploy multiple Atlas servers and one of them will fail since it will be 
> missing its ids.
> 
> To fix this,
> 
> atlas.server.ha.enabled will not be shown on the UI and instead derived. But 
> if for whatever reason we need to do some debugging/troubleshooting on Atlas, 
> we may allow the user to override the property.
> 
> * single Atlas Server => write config with atlas.server.ha.enabled=false (if 
> atlas.server.ha.enabled is set to true, still write it out as false)
> * multiple Atlas Servers => write config with atlas.server.ha.enabled=true 
> (only if atlas.server.ha.enabled is not a property, otherwise, take its value)
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
>  c3af4eb 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py
>  36fb1ea 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/status_params.py
>  0b0d2ae 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.7.0.2.5/configuration/application-properties.xml
>  0fe0827 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
>  b595868 
>   ambari-server/src/test/python/stacks/2.5/common/test_stack_advisor.py 
> 58cce06 
> 
> Diff: https://reviews.apache.org/r/49927/diff/
> 
> 
> Testing
> ---
> 
> Verified following use cases
> 
> 1 Atlas Server: set atlas.server.ha.enabled=false
> 1 Atlas Server with atlas.server.ha.enabled=true, changed 
> atlas.server.ha.enabled=false since doesn't make sense with a single server
> 2 Atlas Servers: set atlas.server.ha.enabled=true
> 2 Atlas Servers with atlas.server.ha.enabled=true, kept 
> atlas.server.ha.enabled=true
> 2 Atlas Servers with atlas.server.ha.enabled=false, kept 
> atlas.server.ha.enabled=false
> 
> mvn clean test
> --
> Total run:1018
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Re: Review Request 49927: Atlas HA fails to come up with error finding ids

2016-07-12 Thread Alejandro Fernandez 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49927/#review141957
---




ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py
 (line 48)


The hostnames are stored in Zookeeper instead of the IDs, so once the Atlas 
servers are restarted they should continue working (in both HA and non-HA 
modes).


- Alejandro Fernandez


On July 11, 2016, 9:42 p.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49927/
> ---
> 
> (Updated July 11, 2016, 9:42 p.m.)
> 
> 
> Review request for Ambari, Madhan Neethiraj, Nahappan Somasundaram, Sumit 
> Mohanty, and Suma Shivaprasad.
> 
> 
> Bugs: AMBARI-17662
> https://issues.apache.org/jira/browse/AMBARI-17662
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Deploy multiple Atlas servers and one of them will fail since it will be 
> missing its ids.
> 
> To fix this,
> 
> atlas.server.ha.enabled will not be shown on the UI and instead derived. But 
> if for whatever reason we need to do some debugging/troubleshooting on Atlas, 
> we may allow the user to override the property.
> 
> * single Atlas Server => write config with atlas.server.ha.enabled=false (if 
> atlas.server.ha.enabled is set to true, still write it out as false)
> * multiple Atlas Servers => write config with atlas.server.ha.enabled=true 
> (only if atlas.server.ha.enabled is not a property, otherwise, take its value)
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
>  c3af4eb 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py
>  36fb1ea 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/status_params.py
>  0b0d2ae 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.7.0.2.5/configuration/application-properties.xml
>  0fe0827 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
>  b595868 
>   ambari-server/src/test/python/stacks/2.5/common/test_stack_advisor.py 
> 58cce06 
> 
> Diff: https://reviews.apache.org/r/49927/diff/
> 
> 
> Testing
> ---
> 
> Verified following use cases
> 
> 1 Atlas Server: set atlas.server.ha.enabled=false
> 1 Atlas Server with atlas.server.ha.enabled=true, changed 
> atlas.server.ha.enabled=false since doesn't make sense with a single server
> 2 Atlas Servers: set atlas.server.ha.enabled=true
> 2 Atlas Servers with atlas.server.ha.enabled=true, kept 
> atlas.server.ha.enabled=true
> 2 Atlas Servers with atlas.server.ha.enabled=false, kept 
> atlas.server.ha.enabled=false
> 
> mvn clean test
> --
> Total run:1018
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Re: Review Request 49953: Storm 1.0 Does Not Support Rolling Upgrades

2016-07-12 Thread Nate Cole 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49953/#review141924
---


Ship it!




Ship It!

- Nate Cole


On July 12, 2016, 10:55 a.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49953/
> ---
> 
> (Updated July 12, 2016, 10:55 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez and Nate Cole.
> 
> 
> Bugs: AMBARI-17667
> https://issues.apache.org/jira/browse/AMBARI-17667
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> HDP 2.5 includes a new version of Storm where packages named 
> {{backtype.storm}} were changed to {{org.apache.storm}}. As a result, Storm 
> local data is not compatible between versions earlier versions of HDP and HDP 
> 2.5. Consider the following situatio where Nimbus and a Supervisor are 
> co-located on the same host:
> 
> - Nimbus deletes local data and restarts on the new version
> - A running 2.4 Supervisor on the same host then re-creates that directory 
> and puts 2.4 data back in
> - When the 2.4 Supervisor goes to upgrade and restart, it can't delete that 
> data again since Nimbus is already running and would stop.
> 
> When starting the Supevisor, the following error is seen:
> ```
> 2016-06-21 23:10:48.000 o.a.s.c.f.s.ConnectionStateManager [INFO] State 
> change: CONNECTED
> 2016-06-21 23:10:48.058 b.s.d.supervisor [INFO] Starting supervisor with id 
> 03d8bceb-0271-4076-810d-04aeaa91533c at host 
> nat-os-r6-omns-dgm10toeriedwngdha-r6-2.openstacklocal
> 2016-06-21 23:10:48.785 b.s.event [ERROR] Error when processing event
> java.lang.RuntimeException: java.lang.ClassNotFoundException: 
> org.apache.storm.generated.LSSupervisorAssignments
> at backtype.storm.utils.LocalState.deserialize(LocalState.java:83) 
> ~[storm-core-0.10.0.2.4.2.0-258.jar:0.10.0.2.4.2.0-258]
> at backtype.storm.utils.LocalState.get(LocalState.java:130) 
> ~[storm-core-0.10.0.2.4.2.0-258.jar:0.10.0.2.4.2.0-258]
> at 
> backtype.storm.local_state$ls_local_assignments.invoke(local_state.clj:83) 
> ~[storm-core-0.10.0.2.4.2.0-258.jar:0.10.0.2.4.2.0-258]
> at 
> backtype.storm.daemon.supervisor$sync_processes.invoke(supervisor.clj:323) 
> ~[storm-core-0.10.0.2.4.2.0-258.jar:0.10.0.2.4.2.0-258]
> ...
> ```
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/CheckDescription.java
>  29feb28 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/StormShutdownWarning.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java 
> 66272e3 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> 85ffb8c 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> 68e8fc2 
>   
> ambari-server/src/test/java/org/apache/ambari/server/checks/StormShutdownWarningTest.java
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/49953/diff/
> 
> 
> Testing
> ---
> 
> Upgraded/Downgrade of Storm from HDP 2.4 to 2.5; verified Supervisors and 
> Nimbus were running and passed SC.
> 
> Unit Tests Pending...
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Re: Review Request 49927: Atlas HA fails to come up with error finding ids

2016-07-12 Thread Sumit Mohanty 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49927/#review141921
---




ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py
 (line 48)


What happens when ATLAS is moved from one host to another?


- Sumit Mohanty


On July 11, 2016, 9:42 p.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49927/
> ---
> 
> (Updated July 11, 2016, 9:42 p.m.)
> 
> 
> Review request for Ambari, Madhan Neethiraj, Nahappan Somasundaram, Sumit 
> Mohanty, and Suma Shivaprasad.
> 
> 
> Bugs: AMBARI-17662
> https://issues.apache.org/jira/browse/AMBARI-17662
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Deploy multiple Atlas servers and one of them will fail since it will be 
> missing its ids.
> 
> To fix this,
> 
> atlas.server.ha.enabled will not be shown on the UI and instead derived. But 
> if for whatever reason we need to do some debugging/troubleshooting on Atlas, 
> we may allow the user to override the property.
> 
> * single Atlas Server => write config with atlas.server.ha.enabled=false (if 
> atlas.server.ha.enabled is set to true, still write it out as false)
> * multiple Atlas Servers => write config with atlas.server.ha.enabled=true 
> (only if atlas.server.ha.enabled is not a property, otherwise, take its value)
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
>  c3af4eb 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py
>  36fb1ea 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/status_params.py
>  0b0d2ae 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.7.0.2.5/configuration/application-properties.xml
>  0fe0827 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
>  b595868 
>   ambari-server/src/test/python/stacks/2.5/common/test_stack_advisor.py 
> 58cce06 
> 
> Diff: https://reviews.apache.org/r/49927/diff/
> 
> 
> Testing
> ---
> 
> Verified following use cases
> 
> 1 Atlas Server: set atlas.server.ha.enabled=false
> 1 Atlas Server with atlas.server.ha.enabled=true, changed 
> atlas.server.ha.enabled=false since doesn't make sense with a single server
> 2 Atlas Servers: set atlas.server.ha.enabled=true
> 2 Atlas Servers with atlas.server.ha.enabled=true, kept 
> atlas.server.ha.enabled=true
> 2 Atlas Servers with atlas.server.ha.enabled=false, kept 
> atlas.server.ha.enabled=false
> 
> mvn clean test
> --
> Total run:1018
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Re: Review Request 49953: Storm 1.0 Does Not Support Rolling Upgrades

2016-07-12 Thread Alejandro Fernandez 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49953/#review141914
---


Ship it!




Ship It!

- Alejandro Fernandez


On July 12, 2016, 2:55 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49953/
> ---
> 
> (Updated July 12, 2016, 2:55 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez and Nate Cole.
> 
> 
> Bugs: AMBARI-17667
> https://issues.apache.org/jira/browse/AMBARI-17667
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> HDP 2.5 includes a new version of Storm where packages named 
> {{backtype.storm}} were changed to {{org.apache.storm}}. As a result, Storm 
> local data is not compatible between versions earlier versions of HDP and HDP 
> 2.5. Consider the following situatio where Nimbus and a Supervisor are 
> co-located on the same host:
> 
> - Nimbus deletes local data and restarts on the new version
> - A running 2.4 Supervisor on the same host then re-creates that directory 
> and puts 2.4 data back in
> - When the 2.4 Supervisor goes to upgrade and restart, it can't delete that 
> data again since Nimbus is already running and would stop.
> 
> When starting the Supevisor, the following error is seen:
> ```
> 2016-06-21 23:10:48.000 o.a.s.c.f.s.ConnectionStateManager [INFO] State 
> change: CONNECTED
> 2016-06-21 23:10:48.058 b.s.d.supervisor [INFO] Starting supervisor with id 
> 03d8bceb-0271-4076-810d-04aeaa91533c at host 
> nat-os-r6-omns-dgm10toeriedwngdha-r6-2.openstacklocal
> 2016-06-21 23:10:48.785 b.s.event [ERROR] Error when processing event
> java.lang.RuntimeException: java.lang.ClassNotFoundException: 
> org.apache.storm.generated.LSSupervisorAssignments
> at backtype.storm.utils.LocalState.deserialize(LocalState.java:83) 
> ~[storm-core-0.10.0.2.4.2.0-258.jar:0.10.0.2.4.2.0-258]
> at backtype.storm.utils.LocalState.get(LocalState.java:130) 
> ~[storm-core-0.10.0.2.4.2.0-258.jar:0.10.0.2.4.2.0-258]
> at 
> backtype.storm.local_state$ls_local_assignments.invoke(local_state.clj:83) 
> ~[storm-core-0.10.0.2.4.2.0-258.jar:0.10.0.2.4.2.0-258]
> at 
> backtype.storm.daemon.supervisor$sync_processes.invoke(supervisor.clj:323) 
> ~[storm-core-0.10.0.2.4.2.0-258.jar:0.10.0.2.4.2.0-258]
> ...
> ```
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/CheckDescription.java
>  29feb28 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/StormShutdownWarning.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java 
> 66272e3 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> 85ffb8c 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> 68e8fc2 
>   
> ambari-server/src/test/java/org/apache/ambari/server/checks/StormShutdownWarningTest.java
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/49953/diff/
> 
> 
> Testing
> ---
> 
> Upgraded/Downgrade of Storm from HDP 2.4 to 2.5; verified Supervisors and 
> Nimbus were running and passed SC.
> 
> Unit Tests Pending...
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Re: Review Request 48309: AMBARI-17047: Firewall check returns WARNING even if iptables and firewalld are stopped on CentOS7

2016-07-12 Thread Masahiro Tanaka 


> On July 12, 2016, 5:02 p.m., Andrew Onischuk wrote:
> > Committed the patch. Please close the reviewboard.

thann you!


- Masahiro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48309/#review141866
---


On July 12, 2016, 3:33 a.m., Masahiro Tanaka wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48309/
> ---
> 
> (Updated July 12, 2016, 3:33 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Andrew Onischuk, Dmytro Sen, 
> Florian Barca, Sumit Mohanty, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-17047
> https://issues.apache.org/jira/browse/AMBARI-17047
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In firewall.py, `systemctl is-active iptables || systemctl is-active 
> firewalld` is passed to `run_in_shell` function, which splits cmd string by 
> using `shlex.split`.
> 
> run_in_shell function finally calls `subprocess.Popen` with `shell=True`, so 
> the cmd string is evaluated like `Popen(['/bin/sh', '-c', 'systemctl', 
> 'is-active', 'iptables', '||', 'systemctl', 'is-active', 'firewalld'])`. This 
> doesn't returns values as expected, because after args[1] (in this case, 
> after the first `is-active`) are evaluated as sh arguements.
> 
> `systemctl is-active` can take multiple arugments, so we can use it.
> 
> 
> Diffs
> -
> 
>   ambari-common/src/main/python/ambari_commons/firewall.py 72e6d26 
>   ambari-server/src/test/python/TestAmbariServer.py 254aeb7 
> 
> Diff: https://reviews.apache.org/r/48309/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test & manual test
> 
> 
> Thanks,
> 
> Masahiro Tanaka
> 
>

Re: Review Request 49935: AMBARI-17633 yarn.nodemanager.remote-app-log-dir should be added stickybit.

2016-07-12 Thread Masahiro Tanaka 


> On July 12, 2016, 3:46 p.m., Andrew Onischuk wrote:
> > Ship It!
> 
> Andrew Onischuk wrote:
> We have tests for those scripts. Are you sure they pass with your changes?

Thank you for reviewing. I'm afraid I didn't check that. I'll check it soon and 
update. Sorry for that.


- Masahiro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49935/#review141861
---


On July 12, 2016, 9:19 a.m., Masahiro Tanaka wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49935/
> ---
> 
> (Updated July 12, 2016, 9:19 a.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, jun aoki, and Myroslav 
> Papirkovskyy.
> 
> 
> Bugs: AMBARI-17633
> https://issues.apache.org/jira/browse/AMBARI-17633
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When installing YARN+MapReduce2, I got a WARN message in nodemanager log(in 
> Log Search View) like below
> ```
> 2016-07-09 06:30:21,865 WARN logaggregation.LogAggregationService 
> LogAggregationService.java:230 - Remote Root Log Dir [/app-logs] already 
> exist, but with incorrect permissions. Expected: [rwxrwxrwt], Found: 
> [rwxrwxrwx]. The cluster may have problems with multiple users. 
> ```
> I think the cause of this WARN is 
> [this](https://github.com/apache/ambari/blob/trunk/ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py#L115).
> We should add stickybit to yarn.nodemanager.remote-app-log-dir.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py
>  ce312e0 
> 
> Diff: https://reviews.apache.org/r/49935/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test && manual test
> 
> I think Hadoop QA result is unrelated to this patch
> 
> 
> Thanks,
> 
> Masahiro Tanaka
> 
>

Re: Review Request 49902: Storm nimbus server fails to come up with CNF backtype.storm.metric.IClusterReporter error

2016-07-12 Thread Dmytro Sen 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49902/
---

(Updated Июль 12, 2016, 4:17 п.п.)


Review request for Ambari, Aravindan Vijayan, Jayush Luniya, and Sid Wagle.


Bugs: AMBARI-17658
https://issues.apache.org/jira/browse/AMBARI-17658


Repository: ambari


Description
---

Correct storm link should be used for non-HDP stacks as well.


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/agent/ExecutionCommand.java
 9c262d9 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java
 6750faf 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
 e0528a0 
  
ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/params_linux.py
 052994d 
  
ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/storm.py
 bce71ca 
  
ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/ui_server.py
 ef2c536 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelperTest.java
 251f00a 
  ambari-server/src/test/python/stacks/2.1/configs/default.json 9a4f8e4 
  ambari-server/src/test/python/stacks/2.1/configs/secured.json ae62dba 

Diff: https://reviews.apache.org/r/49902/diff/


Testing
---

Unit tests passed


Thanks,

Dmytro Sen

Re: Review Request 49898: ACL support for Logsearch Solr

2016-07-12 Thread Oliver Szabo 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49898/
---

(Updated July 12, 2016, 3:21 p.m.)


Review request for Ambari, Miklos Gergely, Robert Levas, Robert Nettleton, 
Sumit Mohanty, and Sebastian Toader.


Changes
---

- increase zk client timeout and defaul solr min/max memory


Bugs: AMBARI-17653
https://issues.apache.org/jira/browse/AMBARI-17653


Repository: ambari


Description
---

Add abillity to use jaas file to authenticate on znode. (that was missing in 
upload configuration to zk)
- Get rid of zkcli of Solr (we can manage everything from solr-client, that 
will be useful, later we can use Watchers with async zookeeper command, which 
can wait until a znode is accessible etc.)
- add new functionalities for solrCloudCli (and solr_cloud_util) like 
setup_kerberos and add_sasl_users, with those we can copy all of the data 
between secure/unsecure znodes (sync...its needed because we have strict 
permissions, but if we disable kerberos we can not change them back), and we 
can create new sasl users. (also it set workd permission on secure node)...in 
the end I not use these features in ambari, using 2 znode (as hbase does) is a 
bit tricky solution. we have some other components where we are not using 
secure zookeeper. if we can find a good solution there later then I can use 
these features (or change them if neededlike, it would be good if we can 
use a zk super user...in that case we do not need to switch between znodes)
- user state json file to track what is the state of the cluster 
(secure/unsecure), that is used to figure it out an enable/disable kerberos 
task happening or there is not state change at all
- remove a lot of unused j2 file from logsearch module
- add retries for atlas/ranger if solr or znode not accessible
- better error messages for timing errors


Diffs (updated)
-

  
ambari-common/src/main/python/resource_management/libraries/functions/solr_cloud_util.py
 4ecddf2 
  ambari-logsearch/ambari-logsearch-logfeeder/src/main/resources/config.json.j2 
3428dd8 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/resources/global.config.json.j2
 cd51118 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/resources/input.config.json.j2
 bc48503 
  ambari-logsearch/ambari-logsearch-logfeeder/src/main/resources/log4j.xml.j2 
4338ee3 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/resources/output.config.json.j2
 63c590e 
  ambari-logsearch/ambari-logsearch-logfeeder/src/main/scripts/run.sh.j2 
713a73a 
  
ambari-logsearch/ambari-logsearch-portal/src/main/configsets/audit_logs/conf/solrconfig.xml.j2
 cb9bea5 
  
ambari-logsearch/ambari-logsearch-portal/src/main/configsets/audit_logs/core.properties.j2
 5185057 
  
ambari-logsearch/ambari-logsearch-portal/src/main/configsets/hadoop_logs/conf/solrconfig.xml.j2
 0152880 
  ambari-logsearch/ambari-logsearch-portal/src/main/resources/log4j.xml.j2 
3b6eac9 
  
ambari-logsearch/ambari-logsearch-portal/src/main/resources/logsearch.properties.j2
 8e3966e 
  ambari-logsearch/ambari-logsearch-portal/src/main/scripts/run.sh.j2 cb78c58 
  ambari-logsearch/ambari-logsearch-solr-client/pom.xml bc3babe 
  
ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/AmbariSolrCloudCLI.java
 1e3cca5 
  
ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/AmbariSolrCloudClient.java
 33e94d0 
  
ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/AmbariSolrCloudClientBuilder.java
 de90859 
  
ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/commands/AbstractStateFileZkCommand.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/commands/AbstractZookeeperConfigCommand.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/commands/AbstractZookeeperRetryCommand.java
 c9e8558 
  
ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/commands/CheckConfigZkCommand.java
 819395c 
  
ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/commands/CheckZnodeZkCommand.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/commands/CopyZnodeZkCommand.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/commands/CreateSaslUsersZkCommand.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/commands/CreateSolrZnodeZkCommand.java
 PRE-CREATION

Review Request 49953: Storm 1.0 Does Not Support Rolling Upgrades

2016-07-12 Thread Jonathan Hurley 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49953/
---

Review request for Ambari, Alejandro Fernandez and Nate Cole.


Bugs: AMBARI-17667
https://issues.apache.org/jira/browse/AMBARI-17667


Repository: ambari


Description
---

HDP 2.5 includes a new version of Storm where packages named {{backtype.storm}} 
were changed to {{org.apache.storm}}. As a result, Storm local data is not 
compatible between versions earlier versions of HDP and HDP 2.5. Consider the 
following situatio where Nimbus and a Supervisor are co-located on the same 
host:

- Nimbus deletes local data and restarts on the new version
- A running 2.4 Supervisor on the same host then re-creates that directory and 
puts 2.4 data back in
- When the 2.4 Supervisor goes to upgrade and restart, it can't delete that 
data again since Nimbus is already running and would stop.

When starting the Supevisor, the following error is seen:
```
2016-06-21 23:10:48.000 o.a.s.c.f.s.ConnectionStateManager [INFO] State change: 
CONNECTED
2016-06-21 23:10:48.058 b.s.d.supervisor [INFO] Starting supervisor with id 
03d8bceb-0271-4076-810d-04aeaa91533c at host 
nat-os-r6-omns-dgm10toeriedwngdha-r6-2.openstacklocal
2016-06-21 23:10:48.785 b.s.event [ERROR] Error when processing event
java.lang.RuntimeException: java.lang.ClassNotFoundException: 
org.apache.storm.generated.LSSupervisorAssignments
at backtype.storm.utils.LocalState.deserialize(LocalState.java:83) 
~[storm-core-0.10.0.2.4.2.0-258.jar:0.10.0.2.4.2.0-258]
at backtype.storm.utils.LocalState.get(LocalState.java:130) 
~[storm-core-0.10.0.2.4.2.0-258.jar:0.10.0.2.4.2.0-258]
at 
backtype.storm.local_state$ls_local_assignments.invoke(local_state.clj:83) 
~[storm-core-0.10.0.2.4.2.0-258.jar:0.10.0.2.4.2.0-258]
at 
backtype.storm.daemon.supervisor$sync_processes.invoke(supervisor.clj:323) 
~[storm-core-0.10.0.2.4.2.0-258.jar:0.10.0.2.4.2.0-258]
...
```


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/checks/CheckDescription.java
 29feb28 
  
ambari-server/src/main/java/org/apache/ambari/server/checks/StormShutdownWarning.java
 PRE-CREATION 
  ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java 
66272e3 
  ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
85ffb8c 
  ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
68e8fc2 
  
ambari-server/src/test/java/org/apache/ambari/server/checks/StormShutdownWarningTest.java
 PRE-CREATION 

Diff: https://reviews.apache.org/r/49953/diff/


Testing
---

Upgraded/Downgrade of Storm from HDP 2.4 to 2.5; verified Supervisors and 
Nimbus were running and passed SC.

Unit Tests Pending...


Thanks,

Jonathan Hurley

Re: Review Request 49898: ACL support for Logsearch Solr

2016-07-12 Thread Sebastian Toader 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49898/#review141874
---


Ship it!




Ship It!

- Sebastian Toader


On July 11, 2016, 6:33 p.m., Oliver Szabo wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49898/
> ---
> 
> (Updated July 11, 2016, 6:33 p.m.)
> 
> 
> Review request for Ambari, Miklos Gergely, Robert Levas, Robert Nettleton, 
> Sumit Mohanty, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-17653
> https://issues.apache.org/jira/browse/AMBARI-17653
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add abillity to use jaas file to authenticate on znode. (that was missing in 
> upload configuration to zk)
> - Get rid of zkcli of Solr (we can manage everything from solr-client, that 
> will be useful, later we can use Watchers with async zookeeper command, which 
> can wait until a znode is accessible etc.)
> - add new functionalities for solrCloudCli (and solr_cloud_util) like 
> setup_kerberos and add_sasl_users, with those we can copy all of the data 
> between secure/unsecure znodes (sync...its needed because we have strict 
> permissions, but if we disable kerberos we can not change them back), and we 
> can create new sasl users. (also it set workd permission on secure node)...in 
> the end I not use these features in ambari, using 2 znode (as hbase does) is 
> a bit tricky solution. we have some other components where we are not using 
> secure zookeeper. if we can find a good solution there later then I can use 
> these features (or change them if neededlike, it would be good if we can 
> use a zk super user...in that case we do not need to switch between znodes)
> - user state json file to track what is the state of the cluster 
> (secure/unsecure), that is used to figure it out an enable/disable kerberos 
> task happening or there is not state change at all
> - remove a lot of unused j2 file from logsearch module
> - add retries for atlas/ranger if solr or znode not accessible
> - better error messages for timing errors
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/solr_cloud_util.py
>  4ecddf2 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/resources/config.json.j2 
> 3428dd8 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/resources/global.config.json.j2
>  cd51118 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/resources/input.config.json.j2
>  bc48503 
>   ambari-logsearch/ambari-logsearch-logfeeder/src/main/resources/log4j.xml.j2 
> 4338ee3 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/resources/output.config.json.j2
>  63c590e 
>   ambari-logsearch/ambari-logsearch-logfeeder/src/main/scripts/run.sh.j2 
> 713a73a 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/configsets/audit_logs/conf/solrconfig.xml.j2
>  cb9bea5 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/configsets/audit_logs/core.properties.j2
>  5185057 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/configsets/hadoop_logs/conf/solrconfig.xml.j2
>  0152880 
>   ambari-logsearch/ambari-logsearch-portal/src/main/resources/log4j.xml.j2 
> 3b6eac9 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/resources/logsearch.properties.j2
>  8e3966e 
>   ambari-logsearch/ambari-logsearch-portal/src/main/scripts/run.sh.j2 cb78c58 
>   ambari-logsearch/ambari-logsearch-solr-client/pom.xml bc3babe 
>   
> ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/AmbariSolrCloudCLI.java
>  1e3cca5 
>   
> ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/AmbariSolrCloudClient.java
>  33e94d0 
>   
> ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/AmbariSolrCloudClientBuilder.java
>  de90859 
>   
> ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/commands/AbstractStateFileZkCommand.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/commands/AbstractZookeeperConfigCommand.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/commands/AbstractZookeeperRetryCommand.java
>  c9e8558 
>   
> ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/commands/CheckConfigZkCommand.java
>  819395c 
>   
> ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/commands/CheckZnodeZkCommand.java
>  PRE-CREATION 
>   
>

Re: Review Request 49947: Atlas server fails to stop after ambari upgrade

2016-07-12 Thread Dmitro Lisnichenko 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49947/#review141872
---


Ship it!




Ship It!

- Dmitro Lisnichenko


On July 12, 2016, 10:20 a.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49947/
> ---
> 
> (Updated July 12, 2016, 10:20 a.m.)
> 
> 
> Review request for Ambari and Dmitro Lisnichenko.
> 
> 
> Bugs: AMBARI-17673
> https://issues.apache.org/jira/browse/AMBARI-17673
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Traceback (most recent call last):
>   File 
> "/var/lib/ambari-agent/cache/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata_server.py",
>  line 171, in 
> MetadataServer().execute()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 280, in execute
> method(env)
>   File 
> "/var/lib/ambari-agent/cache/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata_server.py",
>  line 88, in stop
> import params
>   File 
> "/var/lib/ambari-agent/cache/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py",
>  line 147, in 
> server_host = host + ":" + metadata_port
> TypeError: cannot concatenate 'str' and 'int' objects
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py
>  36fb1ea 
> 
> Diff: https://reviews.apache.org/r/49947/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 48309: AMBARI-17047: Firewall check returns WARNING even if iptables and firewalld are stopped on CentOS7

2016-07-12 Thread Andrew Onischuk 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48309/#review141866
---



Committed the patch. Please close the reviewboard.

- Andrew Onischuk


On July 11, 2016, 6:33 p.m., Masahiro Tanaka wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48309/
> ---
> 
> (Updated July 11, 2016, 6:33 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Andrew Onischuk, Dmytro Sen, 
> Florian Barca, Sumit Mohanty, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-17047
> https://issues.apache.org/jira/browse/AMBARI-17047
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In firewall.py, `systemctl is-active iptables || systemctl is-active 
> firewalld` is passed to `run_in_shell` function, which splits cmd string by 
> using `shlex.split`.
> 
> run_in_shell function finally calls `subprocess.Popen` with `shell=True`, so 
> the cmd string is evaluated like `Popen(['/bin/sh', '-c', 'systemctl', 
> 'is-active', 'iptables', '||', 'systemctl', 'is-active', 'firewalld'])`. This 
> doesn't returns values as expected, because after args[1] (in this case, 
> after the first `is-active`) are evaluated as sh arguements.
> 
> `systemctl is-active` can take multiple arugments, so we can use it.
> 
> 
> Diffs
> -
> 
>   ambari-common/src/main/python/ambari_commons/firewall.py 72e6d26 
>   ambari-server/src/test/python/TestAmbariServer.py 254aeb7 
> 
> Diff: https://reviews.apache.org/r/48309/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test & manual test
> 
> 
> Thanks,
> 
> Masahiro Tanaka
> 
>

Review Request 49947: Atlas server fails to stop after ambari upgrade

2016-07-12 Thread Andrew Onischuk 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49947/
---

Review request for Ambari and Dmitro Lisnichenko.


Bugs: AMBARI-17673
https://issues.apache.org/jira/browse/AMBARI-17673


Repository: ambari


Description
---

Traceback (most recent call last):
  File 
"/var/lib/ambari-agent/cache/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata_server.py",
 line 171, in 
MetadataServer().execute()
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
 line 280, in execute
method(env)
  File 
"/var/lib/ambari-agent/cache/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata_server.py",
 line 88, in stop
import params
  File 
"/var/lib/ambari-agent/cache/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py",
 line 147, in 
server_host = host + ":" + metadata_port
TypeError: cannot concatenate 'str' and 'int' objects


Diffs
-

  
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py
 36fb1ea 

Diff: https://reviews.apache.org/r/49947/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Re: Review Request 49946: Zeppelin service: add default kerberos config for shell interpreter

2016-07-12 Thread Pallav Kulshreshtha 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49946/#review141864
---


Ship it!




Ship It!

- Pallav Kulshreshtha


On July 12, 2016, 6:44 a.m., Renjith Kamath wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49946/
> ---
> 
> (Updated July 12, 2016, 6:44 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, DIPAYAN BHOWMICK, Gaurav 
> Nagar, Pallav Kulshreshtha, Prabhjyot Singh, Rohit Choudhary, and Sumit 
> Mohanty.
> 
> 
> Bugs: AMBARI-17669
> https://issues.apache.org/jira/browse/AMBARI-17669
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> add default kerberos config for shell interpreter
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/master.py
>  88ef49e 
> 
> Diff: https://reviews.apache.org/r/49946/diff/
> 
> 
> Testing
> ---
> 
> manually tested on CentOS 7.0.1
> 
> 
> Thanks,
> 
> Renjith Kamath
> 
>

Re: Review Request 49946: Zeppelin service: add default kerberos config for shell interpreter

2016-07-12 Thread Rohit Choudhary 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49946/#review141863
---


Ship it!




Ship It!

- Rohit Choudhary


On July 12, 2016, 6:44 a.m., Renjith Kamath wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49946/
> ---
> 
> (Updated July 12, 2016, 6:44 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, DIPAYAN BHOWMICK, Gaurav 
> Nagar, Pallav Kulshreshtha, Prabhjyot Singh, Rohit Choudhary, and Sumit 
> Mohanty.
> 
> 
> Bugs: AMBARI-17669
> https://issues.apache.org/jira/browse/AMBARI-17669
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> add default kerberos config for shell interpreter
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/master.py
>  88ef49e 
> 
> Diff: https://reviews.apache.org/r/49946/diff/
> 
> 
> Testing
> ---
> 
> manually tested on CentOS 7.0.1
> 
> 
> Thanks,
> 
> Renjith Kamath
> 
>

Re: Review Request 49935: AMBARI-17633 yarn.nodemanager.remote-app-log-dir should be added stickybit.

2016-07-12 Thread Andrew Onischuk 


> On July 12, 2016, 6:46 a.m., Andrew Onischuk wrote:
> > Ship It!

We have tests for those scripts. Are you sure they pass with your changes?


- Andrew


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49935/#review141861
---


On July 12, 2016, 12:19 a.m., Masahiro Tanaka wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49935/
> ---
> 
> (Updated July 12, 2016, 12:19 a.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, jun aoki, and Myroslav 
> Papirkovskyy.
> 
> 
> Bugs: AMBARI-17633
> https://issues.apache.org/jira/browse/AMBARI-17633
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When installing YARN+MapReduce2, I got a WARN message in nodemanager log(in 
> Log Search View) like below
> ```
> 2016-07-09 06:30:21,865 WARN logaggregation.LogAggregationService 
> LogAggregationService.java:230 - Remote Root Log Dir [/app-logs] already 
> exist, but with incorrect permissions. Expected: [rwxrwxrwt], Found: 
> [rwxrwxrwx]. The cluster may have problems with multiple users. 
> ```
> I think the cause of this WARN is 
> [this](https://github.com/apache/ambari/blob/trunk/ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py#L115).
> We should add stickybit to yarn.nodemanager.remote-app-log-dir.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py
>  ce312e0 
> 
> Diff: https://reviews.apache.org/r/49935/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test && manual test
> 
> I think Hadoop QA result is unrelated to this patch
> 
> 
> Thanks,
> 
> Masahiro Tanaka
> 
>

Re: Review Request 49935: AMBARI-17633 yarn.nodemanager.remote-app-log-dir should be added stickybit.

2016-07-12 Thread Andrew Onischuk 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49935/#review141861
---


Ship it!




Ship It!

- Andrew Onischuk


On July 12, 2016, 12:19 a.m., Masahiro Tanaka wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49935/
> ---
> 
> (Updated July 12, 2016, 12:19 a.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, jun aoki, and Myroslav 
> Papirkovskyy.
> 
> 
> Bugs: AMBARI-17633
> https://issues.apache.org/jira/browse/AMBARI-17633
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When installing YARN+MapReduce2, I got a WARN message in nodemanager log(in 
> Log Search View) like below
> ```
> 2016-07-09 06:30:21,865 WARN logaggregation.LogAggregationService 
> LogAggregationService.java:230 - Remote Root Log Dir [/app-logs] already 
> exist, but with incorrect permissions. Expected: [rwxrwxrwt], Found: 
> [rwxrwxrwx]. The cluster may have problems with multiple users. 
> ```
> I think the cause of this WARN is 
> [this](https://github.com/apache/ambari/blob/trunk/ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py#L115).
> We should add stickybit to yarn.nodemanager.remote-app-log-dir.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py
>  ce312e0 
> 
> Diff: https://reviews.apache.org/r/49935/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test && manual test
> 
> I think Hadoop QA result is unrelated to this patch
> 
> 
> Thanks,
> 
> Masahiro Tanaka
> 
>

Review Request 49946: Zeppelin service: add default kerberos config for shell interpreter

2016-07-12 Thread Renjith Kamath 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49946/
---

Review request for Ambari, Alejandro Fernandez, DIPAYAN BHOWMICK, Gaurav Nagar, 
Pallav Kulshreshtha, Prabhjyot Singh, Rohit Choudhary, and Sumit Mohanty.


Bugs: AMBARI-17669
https://issues.apache.org/jira/browse/AMBARI-17669


Repository: ambari


Description
---

add default kerberos config for shell interpreter


Diffs
-

  
ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/master.py
 88ef49e 

Diff: https://reviews.apache.org/r/49946/diff/


Testing
---

manually tested on CentOS 7.0.1


Thanks,

Renjith Kamath