Re: Review Request 48523: AMBARI-17145 Unformatted configs remain in zeppelin-env.sh

2016-06-15 Thread Jayush Luniya


> On June 10, 2016, 9:35 p.m., Jayush Luniya wrote:
> > Ship It!
> 
> Masahiro Tanaka wrote:
> Thank you! Could you commit this ?

Committed, thanks for your contribution. Can you close the review?


- Jayush


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48523/#review137073
---


On June 9, 2016, 11:49 p.m., Masahiro Tanaka wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48523/
> ---
> 
> (Updated June 9, 2016, 11:49 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Jayush Luniya, Pallav 
> Kulshreshtha, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17145
> https://issues.apache.org/jira/browse/AMBARI-17145
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When we just installed Zeppelin Notebook, /etc/zeppelin/conf/zeppelin-env.sh 
> is like this
> 
> ```
> # Spark master url. eg. spark://master_addr:7077. Leave empty if you want to 
> use local mode
> export MASTER=yarn-client
> export SPARK_YARN_JAR={spark_jar_dir}/zeppelin-spark-0.5.5-SNAPSHOT.jar
> 
> (snip)
> ```
> 
> It maybe 
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/params.py
>  doesn't import from resource_management.libraries.functions.format import 
> format
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/params.py
>  a4efd72 
> 
> Diff: https://reviews.apache.org/r/48523/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test && manual test
> 
> 
> Thanks,
> 
> Masahiro Tanaka
> 
>



Review Request 48734: App timeline Server start fails on enabling HA because namenode is in safemode

2016-06-15 Thread Victor Galgo

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48734/
---

Review request for Ambari, Andriy Babiichuk, Alexandr Antonenko, Andrew 
Onischuk, Dmitro Lisnichenko, Robert Levas, Sandor Magyari, Sumit Mohanty, 
Sebastian Toader, and Yusaku Sako.


Bugs: AMBARI-17182
https://issues.apache.org/jira/browse/AMBARI-17182


Repository: ambari


Description
---

On the last step "Start all" on enabling HA below happens:

Traceback (most recent call last):
File 
"/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/application_timeline_server.py",
 line 147, in 
  ApplicationTimelineServer().execute()
File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
 line 219, in execute
  method(env)
File 
"/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/application_timeline_server.py",
 line 43, in start
  self.configure(env) # FOR SECURITY
File 
"/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/application_timeline_server.py",
 line 54, in configure
  yarn(name='apptimelineserver')
File "/usr/lib/python2.6/site-packages/ambari_commons/os_family_impl.py", 
line 89, in thunk
  return fn(*args, **kwargs)
File 
"/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py",
 line 276, in yarn
  mode=0755
File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", 
line 154, in __init__
  self.env.run()
File 
"/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
line 160, in run
  self.run_action(resource, action)
File 
"/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
line 124, in run_action
  provider_action()
File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py",
 line 463, in action_create_on_execute
  self.action_delayed("create")
File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py",
 line 460, in action_delayed
  self.get_hdfs_resource_executor().action_delayed(action_name, self)
File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py",
 line 259, in action_delayed
  self._set_mode(self.target_status)
File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py",
 line 366, in _set_mode
  self.util.run_command(self.main_resource.resource.target, 
'SETPERMISSION', method='PUT', permission=self.mode, assertable_result=False)
File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py",
 line 195, in run_command
  raise Fail(err_msg)
  resource_management.core.exceptions.Fail: Execution of 'curl -sS -L -w 
'%{http_code}' -X PUT 
'http://testvgalgo.org:50070/webhdfs/v1/ats/done?op=SETPERMISSION=hdfs=755''
 returned status_code=403. 
  {
"RemoteException": {
  "exception": "RetriableException", 
  "javaClassName": "org.apache.hadoop.ipc.RetriableException", 
  "message": "org.apache.hadoop.hdfs.server.namenode.SafeModeException: 
Cannot set permission for /ats/done. Name node is in safe mode.\nThe reported 
blocks 675 needs additional 16 blocks to reach the threshold 0.9900 of total 
blocks 697.\nThe number of live datanodes 20 has reached the minimum number 0. 
Safe mode will be turned off automatically once the thresholds have been 
reached."
}
  }
  
  
This happens because NN is not yet out of safemode at the moment of ats start, 
because DNs just started.

To fix this "stop namenodes" has to be triggered before "start all".

If this is done, on "Start all" it will be ensured that datanodes start prior 
to NN, and that NN are out of safemode before ATS start.


Diffs
-

  
ambari-web/app/controllers/main/admin/highAvailability/nameNode/step9_controller.js
 24677e4 
  ambari-web/app/messages.js 6465812 

Diff: https://reviews.apache.org/r/48734/diff/


Testing
---

Calling set on destroyed view
Calling set on destroyed view
Calling set on destroyed view
Calling set on destroyed view

  28668 tests complete (34 seconds)
  154 tests pending

[INFO] 
[INFO] --- apache-rat-plugin:0.11:check (default) @ ambari-web ---
[INFO] 51 implicit excludes (use -debug for more details).
[INFO] Exclude: .idea/**
[INFO] Exclude: package.json
[INFO] Exclude: public/**
[INFO] Exclude: public-static/**
[INFO] Exclude: app/assets/**
[INFO] Exclude: vendor/**
[INFO] Exclude: node_modules/**
[INFO] Exclude: node/**
[INFO] Exclude: npm-debug.log
[INFO] 1425 resources included (use -debug for more details)
Warning:  org.apache.xerces.jaxp.SAXParserImpl$JAXPSAXParser: Property 
'http://www.oracle.com/xml/jaxp/properties/entityExpansionLimit' is not 

Re: Review Request 48722: Reduce the idle time before first command from next stage is executed on a host

2016-06-15 Thread Sebastian Toader


> On June 15, 2016, 5:54 p.m., Andrew Onischuk wrote:
> > ambari-agent/src/main/python/ambari_agent/Controller.py, line 281
> > 
> >
> > Do we really want to do this only if there are tasks pending. 
> > 
> > Could we do this always? This would make one time operations like start 
> > stop a component decommisions etc. much more responsive.

We could do that? Sumit/Rob/Sandor/Laszlo do you see any issues with that?


- Sebastian


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48722/#review137750
---


On June 15, 2016, 4:55 p.m., Sebastian Toader wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48722/
> ---
> 
> (Updated June 15, 2016, 4:55 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Laszlo Puskas, Robert Levas, 
> Sandor Magyari, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17248
> https://issues.apache.org/jira/browse/AMBARI-17248
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Commands to be executed by ambari-agents are being sent down by the server in 
> the response message to agent heartbeat messages. 
> The server processes the received heartbeat, it checks if there are next 
> commands scheduled to be executed by ambari-agent and adds those to the 
> heartbeat response for the ambari-agent.
> The server organises the commands that can be executed in parallel into 
> stages. Ambari server ensures that only the commands of a single stage is 
> scheduled to be executed by the agent and starts scheduling the commands of 
> the next stage only after all commands of current stage has finished 
> successfully.
> The processing of command status received with the heartbeat message happens 
> asynchronously to heartbeat response in HeartBeatProcessor and 
> ActionScheduler creation thus when the heartbeat response is created the 
> commands for the next stage are not scheduled yet. This means that the next 
> commands will be sent to agent only with the next heartbeat.
> Agents currently sends a heartbeat to the server on command a completion or 
> at a timeout = self.netutil.HEARTBEAT_IDDLE_INTERVAL_SEC – 
> self.netutil.MINIMUM_INTERVAL_BETWEEN_HEARTBEATS interval which is ~10 
> seconds if there are no commands to be executed.
> This means that when the server receives a heartbeat triggered by the 
> completion of the last command from the current stage the server will send 
> the commands for the next stage only 10 seconds later when the next heartbeat 
> is received. This leads to agents spending considerable amount of time idle 
> when there are multiple stages to be executed.
> Agents should heartbeat at a higher rate while there are still pending stages 
> to be executed.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/Controller.py e981a76 
>   ambari-agent/src/main/python/ambari_agent/NetUtil.py 80bf3ae 
>   ambari-agent/src/test/python/ambari_agent/TestNetUtil.py d72e319 
>   ambari-agent/src/test/python/ambari_agent/examples/ControllerTester.py 
> 8103872 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java
>  35a37e3 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatResponse.java
>  1ab7ae9 
>   ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java 
> ac0ddd2 
>   ambari-server/src/main/java/org/apache/ambari/server/state/Clusters.java 
> bd9de13 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
>  3d2388e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClustersImpl.java
>  c26e1e9 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClusterImplTest.java
>  627ade9 
> 
> Diff: https://reviews.apache.org/r/48722/diff/
> 
> 
> Testing
> ---
> 
> Manual testing.
> 
> Unit tests in succeeded.
> 
> 
> Thanks,
> 
> Sebastian Toader
> 
>



Re: Review Request 48722: Reduce the idle time before first command from next stage is executed on a host

2016-06-15 Thread Andrew Onischuk

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48722/#review137750
---




ambari-agent/src/main/python/ambari_agent/Controller.py (line 281)


Do we really want to do this only if there are tasks pending. 

Could we do this always? This would make one time operations like start 
stop a component decommisions etc. much more responsive.


- Andrew Onischuk


On June 15, 2016, 2:55 p.m., Sebastian Toader wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48722/
> ---
> 
> (Updated June 15, 2016, 2:55 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Laszlo Puskas, Robert Levas, 
> Sandor Magyari, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17248
> https://issues.apache.org/jira/browse/AMBARI-17248
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Commands to be executed by ambari-agents are being sent down by the server in 
> the response message to agent heartbeat messages. 
> The server processes the received heartbeat, it checks if there are next 
> commands scheduled to be executed by ambari-agent and adds those to the 
> heartbeat response for the ambari-agent.
> The server organises the commands that can be executed in parallel into 
> stages. Ambari server ensures that only the commands of a single stage is 
> scheduled to be executed by the agent and starts scheduling the commands of 
> the next stage only after all commands of current stage has finished 
> successfully.
> The processing of command status received with the heartbeat message happens 
> asynchronously to heartbeat response in HeartBeatProcessor and 
> ActionScheduler creation thus when the heartbeat response is created the 
> commands for the next stage are not scheduled yet. This means that the next 
> commands will be sent to agent only with the next heartbeat.
> Agents currently sends a heartbeat to the server on command a completion or 
> at a timeout = self.netutil.HEARTBEAT_IDDLE_INTERVAL_SEC – 
> self.netutil.MINIMUM_INTERVAL_BETWEEN_HEARTBEATS interval which is ~10 
> seconds if there are no commands to be executed.
> This means that when the server receives a heartbeat triggered by the 
> completion of the last command from the current stage the server will send 
> the commands for the next stage only 10 seconds later when the next heartbeat 
> is received. This leads to agents spending considerable amount of time idle 
> when there are multiple stages to be executed.
> Agents should heartbeat at a higher rate while there are still pending stages 
> to be executed.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/Controller.py e981a76 
>   ambari-agent/src/main/python/ambari_agent/NetUtil.py 80bf3ae 
>   ambari-agent/src/test/python/ambari_agent/TestNetUtil.py d72e319 
>   ambari-agent/src/test/python/ambari_agent/examples/ControllerTester.py 
> 8103872 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java
>  35a37e3 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatResponse.java
>  1ab7ae9 
>   ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java 
> ac0ddd2 
>   ambari-server/src/main/java/org/apache/ambari/server/state/Clusters.java 
> bd9de13 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
>  3d2388e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClustersImpl.java
>  c26e1e9 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClusterImplTest.java
>  627ade9 
> 
> Diff: https://reviews.apache.org/r/48722/diff/
> 
> 
> Testing
> ---
> 
> Manual testing.
> 
> Unit tests in succeeded.
> 
> 
> Thanks,
> 
> Sebastian Toader
> 
>



Re: Review Request 48722: Reduce the idle time before first command from next stage is executed on a host

2016-06-15 Thread Sandor Magyari

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48722/#review137751
---


Ship it!




Ship It!

- Sandor Magyari


On June 15, 2016, 2:55 p.m., Sebastian Toader wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48722/
> ---
> 
> (Updated June 15, 2016, 2:55 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Laszlo Puskas, Robert Levas, 
> Sandor Magyari, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17248
> https://issues.apache.org/jira/browse/AMBARI-17248
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Commands to be executed by ambari-agents are being sent down by the server in 
> the response message to agent heartbeat messages. 
> The server processes the received heartbeat, it checks if there are next 
> commands scheduled to be executed by ambari-agent and adds those to the 
> heartbeat response for the ambari-agent.
> The server organises the commands that can be executed in parallel into 
> stages. Ambari server ensures that only the commands of a single stage is 
> scheduled to be executed by the agent and starts scheduling the commands of 
> the next stage only after all commands of current stage has finished 
> successfully.
> The processing of command status received with the heartbeat message happens 
> asynchronously to heartbeat response in HeartBeatProcessor and 
> ActionScheduler creation thus when the heartbeat response is created the 
> commands for the next stage are not scheduled yet. This means that the next 
> commands will be sent to agent only with the next heartbeat.
> Agents currently sends a heartbeat to the server on command a completion or 
> at a timeout = self.netutil.HEARTBEAT_IDDLE_INTERVAL_SEC – 
> self.netutil.MINIMUM_INTERVAL_BETWEEN_HEARTBEATS interval which is ~10 
> seconds if there are no commands to be executed.
> This means that when the server receives a heartbeat triggered by the 
> completion of the last command from the current stage the server will send 
> the commands for the next stage only 10 seconds later when the next heartbeat 
> is received. This leads to agents spending considerable amount of time idle 
> when there are multiple stages to be executed.
> Agents should heartbeat at a higher rate while there are still pending stages 
> to be executed.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/Controller.py e981a76 
>   ambari-agent/src/main/python/ambari_agent/NetUtil.py 80bf3ae 
>   ambari-agent/src/test/python/ambari_agent/TestNetUtil.py d72e319 
>   ambari-agent/src/test/python/ambari_agent/examples/ControllerTester.py 
> 8103872 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java
>  35a37e3 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatResponse.java
>  1ab7ae9 
>   ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java 
> ac0ddd2 
>   ambari-server/src/main/java/org/apache/ambari/server/state/Clusters.java 
> bd9de13 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
>  3d2388e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClustersImpl.java
>  c26e1e9 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClusterImplTest.java
>  627ade9 
> 
> Diff: https://reviews.apache.org/r/48722/diff/
> 
> 
> Testing
> ---
> 
> Manual testing.
> 
> Unit tests in succeeded.
> 
> 
> Thanks,
> 
> Sebastian Toader
> 
>



Re: Review Request 48266: Add explicit ambari-server log line indicating cluster creation complete

2016-06-15 Thread Laszlo Puskas

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48266/#review137749
---




ambari-server/src/main/java/org/apache/ambari/server/topology/PersistedStateImpl.java
 (line 149)


Checking the clusterId here is superfluous as the finder should return only 
those entities that have this clusterId.



ambari-server/src/test/java/org/apache/ambari/server/topology/TopologyManagerTest.java
 (line 87)


The member annotated with @TestSubject is instantiated by the framework. 
Why do we need two instances here?


- Laszlo Puskas


On June 15, 2016, 11:33 a.m., Daniel Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48266/
> ---
> 
> (Updated June 15, 2016, 11:33 a.m.)
> 
> 
> Review request for Ambari, Laszlo Puskas, Oliver Szabo, Robert Nettleton, 
> Sandor Magyari, Sumit Mohanty, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-17053
> https://issues.apache.org/jira/browse/AMBARI-17053
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add a log message to see when the cluster is ready to use (cluster creation 
> finishes).
> 
> An event after each heartbeat. At that time cluster provision request is 
> checked if it is finished or not. In case of an ambari server restart, the 
> replayed requests are used to determine which one was the provision request. 
> I could not find a nice way to do it, but I could make a workaround.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionDBAccessorImpl.java
>  8e6fb3f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/events/AmbariEvent.java 
> 1079806 
>   
> ambari-server/src/main/java/org/apache/ambari/server/events/RequestFinishedEvent.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/PersistedState.java
>  77419d8 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/PersistedStateImpl.java
>  324a397 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/TopologyManager.java
>  e3f5b49 
>   
> ambari-server/src/test/java/org/apache/ambari/server/topology/TopologyManagerTest.java
>  fd8653c 
> 
> Diff: https://reviews.apache.org/r/48266/diff/
> 
> 
> Testing
> ---
> 
> Succeeded locally
> 
> 
> Thanks,
> 
> Daniel Gergely
> 
>



Re: Review Request 48659: Fix Spark2 thriftserver Ambari definition bug

2016-06-15 Thread Jayush Luniya


> On June 15, 2016, 4:40 p.m., Jayush Luniya wrote:
> > Ship It!

Committed patch. Please close CR


- Jayush


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48659/#review137762
---


On June 13, 2016, 6:04 p.m., Saisai Shao wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48659/
> ---
> 
> (Updated June 13, 2016, 6:04 p.m.)
> 
> 
> Review request for Ambari and Jayush Luniya.
> 
> 
> Bugs: AMBARI-17204
> https://issues.apache.org/jira/browse/AMBARI-17204
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Fix Spark2 thriftserver Ambari definition bug
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/params.py
>  6925ab5 
> 
> Diff: https://reviews.apache.org/r/48659/diff/
> 
> 
> Testing
> ---
> 
> Unit test is done.
> 
> 
> Thanks,
> 
> Saisai Shao
> 
>



Re: Review Request 48706: 'ambari-server --version' command does not show build number

2016-06-15 Thread Dmitro Lisnichenko

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48706/#review137696
---


Ship it!




Ship It!

- Dmitro Lisnichenko


On June 15, 2016, 11:43 a.m., Vitalyi Brodetskyi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48706/
> ---
> 
> (Updated June 15, 2016, 11:43 a.m.)
> 
> 
> Review request for Ambari, Dmitro Lisnichenko, Dmytro Sen, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17234
> https://issues.apache.org/jira/browse/AMBARI-17234
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> 'ambari-server --version' command does not show build number
> 
> 
> Diffs
> -
> 
>   ambari-server/pom.xml b488971 
>   ambari-server/sbin/ambari-server 83a52c6 
> 
> Diff: https://reviews.apache.org/r/48706/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>



Re: Review Request 48413: Fix misnamed Zookeeper connect strings in Log Search

2016-06-15 Thread Sebastian Toader

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48413/#review137705
---


Ship it!




The patch looks good to me. Also I think it worth making property names string 
costants in the java code.

- Sebastian Toader


On June 15, 2016, 11:24 a.m., Miklos Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48413/
> ---
> 
> (Updated June 15, 2016, 11:24 a.m.)
> 
> 
> Review request for Ambari, Daniel Gergely, Oliver Szabo, Robert Nettleton, 
> Sumit Mohanty, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-17117
> https://issues.apache.org/jira/browse/AMBARI-17117
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Variables/properties holding zookeeper connect strings are misnamed as 
> zk_host, or zk_hosts, which may be misleading. Variable / property names 
> fixed.
> 
> 
> Diffs
> -
> 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/logconfig/FetchConfigFromSolr.java
>  1f86dd0 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/OutputSolr.java
>  6fb0b0e 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/util/SolrUtil.java
>  200a603 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/resources/config.json.j2 
> b6301ca 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/resources/logfeeder.properties
>  076c09c 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/resources/output.config.json.j2
>  a485600 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/test/java/org/apache/ambari/logfeeder/output/OutputSolrTest.java
>  afbccca 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/AuditSolrDao.java
>  03ff0ff 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/ServiceLogsSolrDao.java
>  14125bc 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/SolrDaoBase.java
>  147e148 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/UserConfigSolrDao.java
>  edf1dcc 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/solr/metrics/SolrMetricsLoader.java
>  21c010f 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/resources/logsearch.properties.j2
>  0a94186 
>   
> ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/AmbariSolrCloudCLI.java
>  a2da737 
>   
> ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/AmbariSolrCloudClient.java
>  2805b0b 
>   
> ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/AmbariSolrCloudClientBuilder.java
>  0813221 
>   
> ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/commands/GetSolrHostsCommand.java
>  f814678 
>   
> ambari-logsearch/ambari-logsearch-solr-client/src/test/java/org/apache/ambari/logsearch/solr/AmbariSolrCloudClientTest.java
>  c382c14 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/templates/logfeeder.properties.j2
>  6a52708 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/templates/logsearch.properties.j2
>  9ada5bf 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/templates/output.config.json.j2
>  b31f39b 
>   ambari-server/src/test/python/stacks/2.4/configs/default.json c3aecff 
> 
> Diff: https://reviews.apache.org/r/48413/diff/
> 
> 
> Testing
> ---
> 
> Tested on local cluster
> 
> ambari-logsearch-logfeeder:
> Tests run: 35, Failures: 0, Errors: 0, Skipped: 0
> 
> 
> Thanks,
> 
> Miklos Gergely
> 
>



Review Request 48722: Reduce the idle time before first command from next stage is executed on a host

2016-06-15 Thread Sebastian Toader

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48722/
---

Review request for Ambari, Andrew Onischuk, Laszlo Puskas, Robert Levas, Sandor 
Magyari, and Sumit Mohanty.


Bugs: AMBARI-17248
https://issues.apache.org/jira/browse/AMBARI-17248


Repository: ambari


Description
---

Commands to be executed by ambari-agents are being sent down by the server in 
the response message to agent heartbeat messages. 
The server processes the received heartbeat, it checks if there are next 
commands scheduled to be executed by ambari-agent and adds those to the 
heartbeat response for the ambari-agent.
The server organises the commands that can be executed in parallel into stages. 
Ambari server ensures that only the commands of a single stage is scheduled to 
be executed by the agent and starts scheduling the commands of the next stage 
only after all commands of current stage has finished successfully.
The processing of command status received with the heartbeat message happens 
asynchronously to heartbeat response in HeartBeatProcessor and ActionScheduler 
creation thus when the heartbeat response is created the commands for the next 
stage are not scheduled yet. This means that the next commands will be sent to 
agent only with the next heartbeat.
Agents currently sends a heartbeat to the server on command a completion or at 
a timeout = self.netutil.HEARTBEAT_IDDLE_INTERVAL_SEC – 
self.netutil.MINIMUM_INTERVAL_BETWEEN_HEARTBEATS interval which is ~10 seconds 
if there are no commands to be executed.
This means that when the server receives a heartbeat triggered by the 
completion of the last command from the current stage the server will send the 
commands for the next stage only 10 seconds later when the next heartbeat is 
received. This leads to agents spending considerable amount of time idle when 
there are multiple stages to be executed.
Agents should heartbeat at a higher rate while there are still pending stages 
to be executed.


Diffs
-

  ambari-agent/src/main/python/ambari_agent/Controller.py e981a76 
  ambari-agent/src/main/python/ambari_agent/NetUtil.py 80bf3ae 
  ambari-agent/src/test/python/ambari_agent/TestNetUtil.py d72e319 
  ambari-agent/src/test/python/ambari_agent/examples/ControllerTester.py 
8103872 
  
ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java
 35a37e3 
  
ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatResponse.java
 1ab7ae9 
  ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java 
ac0ddd2 
  ambari-server/src/main/java/org/apache/ambari/server/state/Clusters.java 
bd9de13 
  
ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
 3d2388e 
  
ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClustersImpl.java
 c26e1e9 
  
ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClusterImplTest.java
 627ade9 

Diff: https://reviews.apache.org/r/48722/diff/


Testing
---

Manual testing.

Unit tests in progress.


Thanks,

Sebastian Toader



Re: Review Request 48642: NPE in ambari-server.out when cluster with kerberos is installed

2016-06-15 Thread Daniel Gergely

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48642/
---

(Updated jún. 15, 2016, 8:50 de)


Review request for Ambari, Oliver Szabo, Robert Levas, Sandor Magyari, and 
Sebastian Toader.


Changes
---

Adding comments and skipping log mesage in case of AMBARI_SERVER_ACTION


Bugs: AMBARI-17196
https://issues.apache.org/jira/browse/AMBARI-17196


Repository: ambari


Description
---

When creating a cluster with kerberos, ambari-server.out contains 
NullPointerExceptions, because kerberos related AMBARI_SERVER_ACTIONs do not 
belong to logical tasks.
NPE source is a Long->long conversion when logical task id is null when 
AMBARI_SERVER_ACTION is tried to be retrieved from logicalTaskMap.

The fix is a NPE check that skips the registerPhysicalTaskId method. I added it 
also to the StartHostsTask class.


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/topology/HostRequest.java 
00ecb98 

Diff: https://reviews.apache.org/r/48642/diff/


Testing
---

Changes are done in private classes, so they cannot be accessed from unit tests.


Thanks,

Daniel Gergely



Re: Review Request 48708: Namenode start step failed during EU with RetriableException

2016-06-15 Thread Dmitro Lisnichenko

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48708/#review137699
---


Ship it!




Ship It!

- Dmitro Lisnichenko


On June 15, 2016, 12:33 a.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48708/
> ---
> 
> (Updated June 15, 2016, 12:33 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Dmitro Lisnichenko, and Nate 
> Cole.
> 
> 
> Bugs: AMBARI-17236
> https://issues.apache.org/jira/browse/AMBARI-17236
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When starting NN during an EU, we're hitting this when trying to create HDFS 
> directories:
> ```
> {
>   "RemoteException": {
> "exception": "RetriableException", 
> "javaClassName": "org.apache.hadoop.ipc.RetriableException", 
> "message": "NameNode still not started"
>   }
> }
> ```
> 
> So, the heart of this issue is that, depending on topology and upgrade type, 
> we might not wait for NN to be out of Safe Mode after starting. However, we 
> are always creating directories, regardless of topology/upgrade:
> 
> ```
> # Always run this on non-HA, or active NameNode during HA.
> if is_active_namenode:
>   create_hdfs_directories()
>   create_ranger_audit_hdfs_directories()
> ```
> 
> NameNode, in Safe Mode, is read-only and would forbid this anyway, even if it 
> didn't throw a retryable exception:
> ```
> [hdfs@c6403 root]$ hadoop fs -mkdir /foo
> mkdir: Cannot create directory /foo. Name node is in safe mode.
> ```
> 
> So, it seems like we need to wait for NN to be out of Safe Mode no matter 
> what.
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/resources/hdfs_resource.py
>  18e61fb 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py
>  635f159 
> 
> Diff: https://reviews.apache.org/r/48708/diff/
> 
> 
> Testing
> ---
> 
> PENDING
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>



Re: Review Request 48413: Fix misnamed Zookeeper connect strings in Log Search

2016-06-15 Thread Daniel Gergely

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48413/#review137708
---


Ship it!




Ship It!

- Daniel Gergely


On jún. 15, 2016, 9:24 de, Miklos Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48413/
> ---
> 
> (Updated jún. 15, 2016, 9:24 de)
> 
> 
> Review request for Ambari, Daniel Gergely, Oliver Szabo, Robert Nettleton, 
> Sumit Mohanty, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-17117
> https://issues.apache.org/jira/browse/AMBARI-17117
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Variables/properties holding zookeeper connect strings are misnamed as 
> zk_host, or zk_hosts, which may be misleading. Variable / property names 
> fixed.
> 
> 
> Diffs
> -
> 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/logconfig/FetchConfigFromSolr.java
>  1f86dd0 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/OutputSolr.java
>  6fb0b0e 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/util/SolrUtil.java
>  200a603 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/resources/config.json.j2 
> b6301ca 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/resources/logfeeder.properties
>  076c09c 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/resources/output.config.json.j2
>  a485600 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/test/java/org/apache/ambari/logfeeder/output/OutputSolrTest.java
>  afbccca 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/AuditSolrDao.java
>  03ff0ff 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/ServiceLogsSolrDao.java
>  14125bc 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/SolrDaoBase.java
>  147e148 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/UserConfigSolrDao.java
>  edf1dcc 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/solr/metrics/SolrMetricsLoader.java
>  21c010f 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/resources/logsearch.properties.j2
>  0a94186 
>   
> ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/AmbariSolrCloudCLI.java
>  a2da737 
>   
> ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/AmbariSolrCloudClient.java
>  2805b0b 
>   
> ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/AmbariSolrCloudClientBuilder.java
>  0813221 
>   
> ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/commands/GetSolrHostsCommand.java
>  f814678 
>   
> ambari-logsearch/ambari-logsearch-solr-client/src/test/java/org/apache/ambari/logsearch/solr/AmbariSolrCloudClientTest.java
>  c382c14 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/templates/logfeeder.properties.j2
>  6a52708 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/templates/logsearch.properties.j2
>  9ada5bf 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/templates/output.config.json.j2
>  b31f39b 
>   ambari-server/src/test/python/stacks/2.4/configs/default.json c3aecff 
> 
> Diff: https://reviews.apache.org/r/48413/diff/
> 
> 
> Testing
> ---
> 
> Tested on local cluster
> 
> ambari-logsearch-logfeeder:
> Tests run: 35, Failures: 0, Errors: 0, Skipped: 0
> 
> 
> Thanks,
> 
> Miklos Gergely
> 
>



Re: Review Request 48706: 'ambari-server --version' command does not show build number

2016-06-15 Thread Vitalyi Brodetskyi

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48706/
---

(Updated Червень 15, 2016, 8:43 до полудня)


Review request for Ambari, Dmitro Lisnichenko, Dmytro Sen, and Sumit Mohanty.


Bugs: AMBARI-17234
https://issues.apache.org/jira/browse/AMBARI-17234


Repository: ambari


Description
---

'ambari-server --version' command does not show build number


Diffs
-

  ambari-server/pom.xml b488971 
  ambari-server/sbin/ambari-server 83a52c6 

Diff: https://reviews.apache.org/r/48706/diff/


Testing
---


Thanks,

Vitalyi Brodetskyi



Re: Review Request 48706: 'ambari-server --version' command does not show build number

2016-06-15 Thread Dmytro Sen

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48706/#review137693
---


Ship it!




Ship It!

- Dmytro Sen


On Июнь 15, 2016, 8:43 д.п., Vitalyi Brodetskyi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48706/
> ---
> 
> (Updated Июнь 15, 2016, 8:43 д.п.)
> 
> 
> Review request for Ambari, Dmitro Lisnichenko, Dmytro Sen, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17234
> https://issues.apache.org/jira/browse/AMBARI-17234
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> 'ambari-server --version' command does not show build number
> 
> 
> Diffs
> -
> 
>   ambari-server/pom.xml b488971 
>   ambari-server/sbin/ambari-server 83a52c6 
> 
> Diff: https://reviews.apache.org/r/48706/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>



Re: Review Request 48266: Add explicit ambari-server log line indicating cluster creation complete

2016-06-15 Thread Sebastian Toader

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48266/#review137713
---


Ship it!




Ship It!

- Sebastian Toader


On June 15, 2016, 1:33 p.m., Daniel Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48266/
> ---
> 
> (Updated June 15, 2016, 1:33 p.m.)
> 
> 
> Review request for Ambari, Laszlo Puskas, Oliver Szabo, Robert Nettleton, 
> Sandor Magyari, Sumit Mohanty, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-17053
> https://issues.apache.org/jira/browse/AMBARI-17053
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add a log message to see when the cluster is ready to use (cluster creation 
> finishes).
> 
> An event after each heartbeat. At that time cluster provision request is 
> checked if it is finished or not. In case of an ambari server restart, the 
> replayed requests are used to determine which one was the provision request. 
> I could not find a nice way to do it, but I could make a workaround.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionDBAccessorImpl.java
>  8e6fb3f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/events/AmbariEvent.java 
> 1079806 
>   
> ambari-server/src/main/java/org/apache/ambari/server/events/RequestFinishedEvent.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/PersistedState.java
>  77419d8 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/PersistedStateImpl.java
>  324a397 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/TopologyManager.java
>  e3f5b49 
>   
> ambari-server/src/test/java/org/apache/ambari/server/topology/TopologyManagerTest.java
>  fd8653c 
> 
> Diff: https://reviews.apache.org/r/48266/diff/
> 
> 
> Testing
> ---
> 
> Succeeded locally
> 
> 
> Thanks,
> 
> Daniel Gergely
> 
>



Review Request 48725: Supervisor start failed after Ambari upgrade (intermittent)

2016-06-15 Thread Andrew Onischuk

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48725/
---

Review request for Ambari and Dmitro Lisnichenko.


Bugs: AMBARI-17252
https://issues.apache.org/jira/browse/AMBARI-17252


Repository: ambari


Description
---

ambari-server --hash  
6cd42e12348dbf0166428bc8b701586bb2c7b0bb  
Build # - 2.4.0.0-653

**Steps**

  1. Deploy HDP-2.4.2.0 cluster with Ambari 2.2.2.0
  2. Upgrade Ambari to 2.4.0.0
  3. Stop all services
  4. Start all services

Observed below error for Supervisor start




Traceback (most recent call last):
  File 
"/var/lib/ambari-agent/cache/common-services/STORM/0.9.1/package/scripts/supervisor.py",
 line 112, in 
Supervisor().execute()
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
 line 257, in execute
method(env)
  File 
"/var/lib/ambari-agent/cache/common-services/STORM/0.9.1/package/scripts/supervisor.py",
 line 88, in start
service("supervisor", action="start")
  File 
"/var/lib/ambari-agent/cache/common-services/STORM/0.9.1/package/scripts/service.py",
 line 75, in service
path = params.storm_bin_dir)
  File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", 
line 155, in __init__
self.env.run()
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
line 160, in run
self.run_action(resource, action)
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
line 124, in run_action
provider_action()
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py",
 line 273, in action_run
tries=self.resource.tries, try_sleep=self.resource.try_sleep)
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 70, 
in inner
result = function(command, **kwargs)
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 92, 
in checked_call
tries=tries, try_sleep=try_sleep)
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 140, 
in _call_wrapper
result = _call(command, **kwargs_copy)
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 293, 
in _call
raise Fail(err_msg)
resource_management.core.exceptions.Fail: Execution of 
'/usr/jdk64/jdk1.8.0_77/bin/jps -l  | grep storm.daemon.supervisor$ && 
/usr/jdk64/jdk1.8.0_77/bin/jps -l  | grep storm.daemon.supervisor$ | awk 
{'print $1'} > /var/run/storm/supervisor.pid' returned 1.  Hortonworks 
#
This is MOTD message, added for testing in qe infra


Note: Afterwards tried to manually start Storm and Supervisor came up fine

Logs attached


Diffs
-

  ambari-common/src/main/python/resource_management/core/shell.py fac264c 
  
ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/service.py
 6da159a 
  ambari-server/src/test/python/stacks/2.1/STORM/test_storm_drpc_server.py 
fba7b9c 
  ambari-server/src/test/python/stacks/2.1/STORM/test_storm_nimbus.py 0f95a0c 
  ambari-server/src/test/python/stacks/2.1/STORM/test_storm_rest_api_service.py 
0fe57cf 
  ambari-server/src/test/python/stacks/2.1/STORM/test_storm_supervisor.py 
c70e06c 
  ambari-server/src/test/python/stacks/2.1/STORM/test_storm_supervisor_prod.py 
f4f6bae 
  ambari-server/src/test/python/stacks/2.1/STORM/test_storm_ui_server.py 
101fbe6 

Diff: https://reviews.apache.org/r/48725/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk



Re: Review Request 48725: Supervisor start failed after Ambari upgrade (intermittent)

2016-06-15 Thread Dmitro Lisnichenko

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48725/#review137719
---


Ship it!




Ship It!

- Dmitro Lisnichenko


On June 15, 2016, 3:14 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48725/
> ---
> 
> (Updated June 15, 2016, 3:14 p.m.)
> 
> 
> Review request for Ambari and Dmitro Lisnichenko.
> 
> 
> Bugs: AMBARI-17252
> https://issues.apache.org/jira/browse/AMBARI-17252
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> ambari-server --hash  
> 6cd42e12348dbf0166428bc8b701586bb2c7b0bb  
> Build # - 2.4.0.0-653
> 
> **Steps**
> 
>   1. Deploy HDP-2.4.2.0 cluster with Ambari 2.2.2.0
>   2. Upgrade Ambari to 2.4.0.0
>   3. Stop all services
>   4. Start all services
> 
> Observed below error for Supervisor start
> 
> 
> 
> 
> Traceback (most recent call last):
>   File 
> "/var/lib/ambari-agent/cache/common-services/STORM/0.9.1/package/scripts/supervisor.py",
>  line 112, in 
> Supervisor().execute()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 257, in execute
> method(env)
>   File 
> "/var/lib/ambari-agent/cache/common-services/STORM/0.9.1/package/scripts/supervisor.py",
>  line 88, in start
> service("supervisor", action="start")
>   File 
> "/var/lib/ambari-agent/cache/common-services/STORM/0.9.1/package/scripts/service.py",
>  line 75, in service
> path = params.storm_bin_dir)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 
> 155, in __init__
> self.env.run()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
> line 160, in run
> self.run_action(resource, action)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
> line 124, in run_action
> provider_action()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py",
>  line 273, in action_run
> tries=self.resource.tries, try_sleep=self.resource.try_sleep)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
> 70, in inner
> result = function(command, **kwargs)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
> 92, in checked_call
> tries=tries, try_sleep=try_sleep)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
> 140, in _call_wrapper
> result = _call(command, **kwargs_copy)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
> 293, in _call
> raise Fail(err_msg)
> resource_management.core.exceptions.Fail: Execution of 
> '/usr/jdk64/jdk1.8.0_77/bin/jps -l  | grep storm.daemon.supervisor$ && 
> /usr/jdk64/jdk1.8.0_77/bin/jps -l  | grep storm.daemon.supervisor$ | awk 
> {'print $1'} > /var/run/storm/supervisor.pid' returned 1.  
> Hortonworks #
> This is MOTD message, added for testing in qe infra
> 
> 
> Note: Afterwards tried to manually start Storm and Supervisor came up fine
> 
> Logs attached
> 
> 
> Diffs
> -
> 
>   ambari-common/src/main/python/resource_management/core/shell.py fac264c 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/service.py
>  6da159a 
>   ambari-server/src/test/python/stacks/2.1/STORM/test_storm_drpc_server.py 
> fba7b9c 
>   ambari-server/src/test/python/stacks/2.1/STORM/test_storm_nimbus.py 0f95a0c 
>   
> ambari-server/src/test/python/stacks/2.1/STORM/test_storm_rest_api_service.py 
> 0fe57cf 
>   ambari-server/src/test/python/stacks/2.1/STORM/test_storm_supervisor.py 
> c70e06c 
>   
> ambari-server/src/test/python/stacks/2.1/STORM/test_storm_supervisor_prod.py 
> f4f6bae 
>   ambari-server/src/test/python/stacks/2.1/STORM/test_storm_ui_server.py 
> 101fbe6 
> 
> Diff: https://reviews.apache.org/r/48725/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>



Review Request 48726: AMBARI-17250 : Use right principals for Hbase Master in Kerberos enabled Ranger Hbase Plugin

2016-06-15 Thread Gautam Borad

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48726/
---

Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Srimanth 
Gunturi, and Velmurugan Periasamy.


Bugs: AMBARI-17250
https://issues.apache.org/jira/browse/AMBARI-17250


Repository: ambari


Description
---

**Issue :**
Hbase test connection and resource lookup fails on secure kerberized cluster 
due to invalid principal for Hbase Master. 
Ambari configs are passing hbase_jaas_principal need to correct that.


Diffs
-

  
ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/configuration/ranger-atlas-plugin-properties.xml
 f3bdc2a 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
614f0b1 

Diff: https://reviews.apache.org/r/48726/diff/


Testing
---

Verified recommendations for audit to solr as well as hdfs for Atlas (if 
enabled from Ranger configs)


Thanks,

Gautam Borad



Re: Review Request 48549: AMBARI-17165 Handle Java patches execution during Ranger upgrade

2016-06-15 Thread Mugdha Varadkar

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48549/
---

(Updated June 15, 2016, 12:41 p.m.)


Review request for Ambari, Gautam Borad, Jonathan Hurley, Nate Cole, Srimanth 
Gunturi, and Velmurugan Periasamy.


Changes
---

Updating execution of java patches to pre-upgrade block


Bugs: AMBARI-17165
https://issues.apache.org/jira/browse/AMBARI-17165


Repository: ambari


Description
---

Revisiting execution of java patches during upgrade for Ranger Service


Diffs (updated)
-

  
ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/params.py
 29ac561 
  
ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py
 b86a09b 
  
ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.3.xml
 4fd5801 
  
ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.4.xml
 272a3cc 
  ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.3.xml 
b0cff68 
  ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.4.xml 
0b72254 
  
ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.3.xml
 111b432 
  
ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.4.xml
 9365646 
  
ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml
 464c444 
  ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.3.xml 
712241b 
  ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.4.xml 
4187d64 
  ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
dd04b64 
  
ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.4.xml
 e83b54b 
  
ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml
 27461e8 
  ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.4.xml 
4065e87 
  ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
ad1bc34 
  
ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.5.xml
 460e6b3 
  ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.5.xml 
6ce4c81 

Diff: https://reviews.apache.org/r/48549/diff/


Testing
---

Tested Ranger upgrade from stack 2.4 to 2.5


Thanks,

Mugdha Varadkar



Re: Review Request 48494: Implement config values trimming for deployment via blueprint

2016-06-15 Thread Sebastian Toader

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48494/#review137714
---


Ship it!




Ship It!

- Sebastian Toader


On June 14, 2016, 3:45 p.m., Dmytro Sen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48494/
> ---
> 
> (Updated June 14, 2016, 3:45 p.m.)
> 
> 
> Review request for Ambari, Robert Nettleton, Sebastian Toader, and Vitalyi 
> Brodetskyi.
> 
> 
> Bugs: AMBARI-17146
> https://issues.apache.org/jira/browse/AMBARI-17146
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Implement config values trimming for deployment via blueprint as we do in UI
> 
>   trimProperty: function (property) {
> var displayType = Em.get(property, 'displayType');
> var value = Em.get(property, 'value');
> var name = Em.get(property, 'name');
> var rez;
> switch (displayType) {
>   case 'directories':
>   case 'directory':
> rez = value.replace(/,/g, ' ').trim().split(/\s+/g).join(',');
> break;
>   case 'host':
> rez = value.trim();
> break;
>   case 'password':
> break;
>   default:
> if (name == 'javax.jdo.option.ConnectionURL' || name == 
> 'oozie.service.JPAService.jdbc.url') {
>   rez = value.trim();
> }
> rez = (typeof value == 'string') ? value.replace(/(\s+$)/g, '') : 
> value;
> }
> return ((rez == '') || (rez == undefined)) ? value : rez;
>   },
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
>  9094698 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/DefaultTrimmingStrategy.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/DeleteSpacesAtTheEndTrimmingStrategy.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/DirectoriesTrimmingStrategy.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/PasswordTrimmingStrategy.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/PropertyValueTrimmingStrategyDefiner.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/Stack.java
>  ad8d4f9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/TrimmingStrategy.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
>  cda8fb8 
> 
> Diff: https://reviews.apache.org/r/48494/diff/
> 
> 
> Testing
> ---
> 
> Unit tests and manual tests passed
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>



Re: Review Request 48494: Implement config values trimming for deployment via blueprint

2016-06-15 Thread Daniel Gergely

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48494/#review137716
---




ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
 (lines 364 - 367)


Since there is no computational expensive calculation in this block, the if 
statement can be omitted. isDebugEnabled is used when logging the message needs 
some calculation that is used by the debug logging only.



ambari-server/src/main/java/org/apache/ambari/server/controller/internal/DirectoriesTrimmingStrategy.java
 (line 25)


Is it possible that an element in the comma separated string contains space?

If it is so, then maybe this whole function could be replaced by:
return stringToTrim.replaceAll("\\s*,+\\s*", ",");
(replacing sequences containing at least one comma and spaces around them 
to a single comma. It also removes trailing and leading commas, but preserves 
spaces within the elements. You can try it on http://regex101.com)



ambari-server/src/main/java/org/apache/ambari/server/controller/internal/DirectoriesTrimmingStrategy.java
 (lines 26 - 33)


If the regex in my previuos comment cannot be used, then you can use 
StringUtils to simplify this part:
return StringUtils.join(stringsArray, ",");



ambari-server/src/main/java/org/apache/ambari/server/controller/internal/PropertyValueTrimmingStrategyDefiner.java
 (line 41)


Simplify this by swapping the constant and the variable (you can use it as 
a thumbrule)
if("directory".equals(type) || "directories".equals(type))



ambari-server/src/main/java/org/apache/ambari/server/controller/internal/PropertyValueTrimmingStrategyDefiner.java
 (line 43)


Swap constant and variable


- Daniel Gergely


On jún. 14, 2016, 1:45 du, Dmytro Sen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48494/
> ---
> 
> (Updated jún. 14, 2016, 1:45 du)
> 
> 
> Review request for Ambari, Robert Nettleton, Sebastian Toader, and Vitalyi 
> Brodetskyi.
> 
> 
> Bugs: AMBARI-17146
> https://issues.apache.org/jira/browse/AMBARI-17146
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Implement config values trimming for deployment via blueprint as we do in UI
> 
>   trimProperty: function (property) {
> var displayType = Em.get(property, 'displayType');
> var value = Em.get(property, 'value');
> var name = Em.get(property, 'name');
> var rez;
> switch (displayType) {
>   case 'directories':
>   case 'directory':
> rez = value.replace(/,/g, ' ').trim().split(/\s+/g).join(',');
> break;
>   case 'host':
> rez = value.trim();
> break;
>   case 'password':
> break;
>   default:
> if (name == 'javax.jdo.option.ConnectionURL' || name == 
> 'oozie.service.JPAService.jdbc.url') {
>   rez = value.trim();
> }
> rez = (typeof value == 'string') ? value.replace(/(\s+$)/g, '') : 
> value;
> }
> return ((rez == '') || (rez == undefined)) ? value : rez;
>   },
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
>  9094698 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/DefaultTrimmingStrategy.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/DeleteSpacesAtTheEndTrimmingStrategy.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/DirectoriesTrimmingStrategy.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/PasswordTrimmingStrategy.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/PropertyValueTrimmingStrategyDefiner.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/Stack.java
>  ad8d4f9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/TrimmingStrategy.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
>  cda8fb8 
> 
> Diff: https://reviews.apache.org/r/48494/diff/
> 
> 
> Testing
> ---
> 
> Unit tests and manual tests passed
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>



Re: Review Request 48266: Add explicit ambari-server log line indicating cluster creation complete

2016-06-15 Thread Daniel Gergely

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48266/
---

(Updated jún. 15, 2016, 11:33 de)


Review request for Ambari, Laszlo Puskas, Oliver Szabo, Robert Nettleton, 
Sandor Magyari, Sumit Mohanty, and Sebastian Toader.


Changes
---

Moving event posting from HeartbeatHandler to ActionDBAccessor


Bugs: AMBARI-17053
https://issues.apache.org/jira/browse/AMBARI-17053


Repository: ambari


Description
---

Add a log message to see when the cluster is ready to use (cluster creation 
finishes).

An event after each heartbeat. At that time cluster provision request is 
checked if it is finished or not. In case of an ambari server restart, the 
replayed requests are used to determine which one was the provision request. I 
could not find a nice way to do it, but I could make a workaround.


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionDBAccessorImpl.java
 8e6fb3f 
  ambari-server/src/main/java/org/apache/ambari/server/events/AmbariEvent.java 
1079806 
  
ambari-server/src/main/java/org/apache/ambari/server/events/RequestFinishedEvent.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/topology/PersistedState.java
 77419d8 
  
ambari-server/src/main/java/org/apache/ambari/server/topology/PersistedStateImpl.java
 324a397 
  
ambari-server/src/main/java/org/apache/ambari/server/topology/TopologyManager.java
 e3f5b49 
  
ambari-server/src/test/java/org/apache/ambari/server/topology/TopologyManagerTest.java
 fd8653c 

Diff: https://reviews.apache.org/r/48266/diff/


Testing
---

Succeeded locally


Thanks,

Daniel Gergely



Re: Review Request 48642: NPE in ambari-server.out when cluster with kerberos is installed

2016-06-15 Thread Sebastian Toader

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48642/#review137710
---


Ship it!




Ship It!

- Sebastian Toader


On June 15, 2016, 1:41 p.m., Daniel Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48642/
> ---
> 
> (Updated June 15, 2016, 1:41 p.m.)
> 
> 
> Review request for Ambari, Oliver Szabo, Robert Levas, Sandor Magyari, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-17196
> https://issues.apache.org/jira/browse/AMBARI-17196
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When creating a cluster with kerberos, ambari-server.out contains 
> NullPointerExceptions, because kerberos related AMBARI_SERVER_ACTIONs do not 
> belong to logical tasks.
> NPE source is a Long->long conversion when logical task id is null when 
> AMBARI_SERVER_ACTION is tried to be retrieved from logicalTaskMap.
> 
> The fix is a NPE check that skips the registerPhysicalTaskId method. I added 
> it also to the StartHostsTask class.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/HostRequest.java
>  00ecb98 
> 
> Diff: https://reviews.apache.org/r/48642/diff/
> 
> 
> Testing
> ---
> 
> Changes are done in private classes, so they cannot be accessed from unit 
> tests.
> 
> 
> Thanks,
> 
> Daniel Gergely
> 
>



Re: Review Request 48642: NPE in ambari-server.out when cluster with kerberos is installed

2016-06-15 Thread Daniel Gergely

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48642/
---

(Updated jún. 15, 2016, 11:41 de)


Review request for Ambari, Oliver Szabo, Robert Levas, Sandor Magyari, and 
Sebastian Toader.


Changes
---

Comment and log message fix


Bugs: AMBARI-17196
https://issues.apache.org/jira/browse/AMBARI-17196


Repository: ambari


Description
---

When creating a cluster with kerberos, ambari-server.out contains 
NullPointerExceptions, because kerberos related AMBARI_SERVER_ACTIONs do not 
belong to logical tasks.
NPE source is a Long->long conversion when logical task id is null when 
AMBARI_SERVER_ACTION is tried to be retrieved from logicalTaskMap.

The fix is a NPE check that skips the registerPhysicalTaskId method. I added it 
also to the StartHostsTask class.


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/topology/HostRequest.java 
00ecb98 

Diff: https://reviews.apache.org/r/48642/diff/


Testing
---

Changes are done in private classes, so they cannot be accessed from unit tests.


Thanks,

Daniel Gergely



Re: Review Request 48722: Reduce the idle time before first command from next stage is executed on a host

2016-06-15 Thread Robert Levas

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48722/#review137717
---


Ship it!




Ship It!

- Robert Levas


On June 15, 2016, 5:46 a.m., Sebastian Toader wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48722/
> ---
> 
> (Updated June 15, 2016, 5:46 a.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Laszlo Puskas, Robert Levas, 
> Sandor Magyari, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17248
> https://issues.apache.org/jira/browse/AMBARI-17248
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Commands to be executed by ambari-agents are being sent down by the server in 
> the response message to agent heartbeat messages. 
> The server processes the received heartbeat, it checks if there are next 
> commands scheduled to be executed by ambari-agent and adds those to the 
> heartbeat response for the ambari-agent.
> The server organises the commands that can be executed in parallel into 
> stages. Ambari server ensures that only the commands of a single stage is 
> scheduled to be executed by the agent and starts scheduling the commands of 
> the next stage only after all commands of current stage has finished 
> successfully.
> The processing of command status received with the heartbeat message happens 
> asynchronously to heartbeat response in HeartBeatProcessor and 
> ActionScheduler creation thus when the heartbeat response is created the 
> commands for the next stage are not scheduled yet. This means that the next 
> commands will be sent to agent only with the next heartbeat.
> Agents currently sends a heartbeat to the server on command a completion or 
> at a timeout = self.netutil.HEARTBEAT_IDDLE_INTERVAL_SEC – 
> self.netutil.MINIMUM_INTERVAL_BETWEEN_HEARTBEATS interval which is ~10 
> seconds if there are no commands to be executed.
> This means that when the server receives a heartbeat triggered by the 
> completion of the last command from the current stage the server will send 
> the commands for the next stage only 10 seconds later when the next heartbeat 
> is received. This leads to agents spending considerable amount of time idle 
> when there are multiple stages to be executed.
> Agents should heartbeat at a higher rate while there are still pending stages 
> to be executed.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/Controller.py e981a76 
>   ambari-agent/src/main/python/ambari_agent/NetUtil.py 80bf3ae 
>   ambari-agent/src/test/python/ambari_agent/TestNetUtil.py d72e319 
>   ambari-agent/src/test/python/ambari_agent/examples/ControllerTester.py 
> 8103872 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java
>  35a37e3 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatResponse.java
>  1ab7ae9 
>   ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java 
> ac0ddd2 
>   ambari-server/src/main/java/org/apache/ambari/server/state/Clusters.java 
> bd9de13 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
>  3d2388e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClustersImpl.java
>  c26e1e9 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClusterImplTest.java
>  627ade9 
> 
> Diff: https://reviews.apache.org/r/48722/diff/
> 
> 
> Testing
> ---
> 
> Manual testing.
> 
> Unit tests in progress.
> 
> 
> Thanks,
> 
> Sebastian Toader
> 
>



Re: Review Request 48702: Add ability to set GET request directives

2016-06-15 Thread Nate Cole

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48702/#review137720
---


Ship it!




Ship It!

- Nate Cole


On June 14, 2016, 7:12 p.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48702/
> ---
> 
> (Updated June 14, 2016, 7:12 p.m.)
> 
> 
> Review request for Ambari, Ajit Kumar, Di Li, Jonathan Hurley, and Nate Cole.
> 
> 
> Bugs: AMBARI-17224
> https://issues.apache.org/jira/browse/AMBARI-17224
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add ability to set GET request directives so that directives may be specified 
> in a GET requests. For example, to limit or increase the amount of work 
> needed to be performed internally to generate the data being returned.
> 
> Most data in GET requests are retrieved but filtered from the response based 
> on the predicate. This means that any data the requires significant 
> processing to calculate will be calculated regardless of the requested 
> fields. By using a directive this can be controlled.
> 
> Note: It may be possible the use of a PropertyProvider to control this 
> instead, but that may not be desired in all cases.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/handlers/ReadHandler.java
>  95e45d6 
>   ambari-server/src/main/java/org/apache/ambari/server/api/query/Query.java 
> fff842a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/query/QueryImpl.java 
> 18fd94b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/BaseResourceDefinition.java
>  dcc5217 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/ResourceDefinition.java
>  8df1a02 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/SimpleResourceDefinition.java
>  394e295 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/UpgradeResourceDefinition.java
>  bf7860b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/BaseRequest.java
>  0ebeb19 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/RequestFactory.java
>  19fee8e 
>   
> ambari-server/src/test/java/org/apache/ambari/server/api/handlers/ReadHandlerTest.java
>  5cb601e 
>   
> ambari-server/src/test/java/org/apache/ambari/server/api/resources/BaseResourceDefinitionTest.java
>  f174cb6 
>   
> ambari-server/src/test/java/org/apache/ambari/server/api/resources/SimpleResourceDefinitionTest.java
>  6169ee7 
>   
> ambari-server/src/test/java/org/apache/ambari/server/api/services/RequestFactoryTest.java
>  11568c9 
>   
> ambari-server/src/test/java/org/apache/ambari/server/audit/request/creator/AuditEventCreatorTestHelper.java
>  2642418 
> 
> Diff: https://reviews.apache.org/r/48702/diff/
> 
> 
> Testing
> ---
> 
> # Local test results: 
> 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 1:25:36.424s
> [INFO] Finished at: Tue Jun 14 14:53:08 EDT 2016
> [INFO] Final Memory: 59M/1864M
> [INFO] 
> 
> 
> # Jenkins test results: PENDING
> 
> 
> Thanks,
> 
> Robert Levas
> 
>



Re: Review Request 48726: AMBARI-17247 : Populate audit to solr / hdfs properties for Atlas

2016-06-15 Thread Gautam Borad

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48726/
---

(Updated June 15, 2016, 12:42 p.m.)


Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Srimanth 
Gunturi, and Velmurugan Periasamy.


Changes
---

Corrected BUG details


Summary (updated)
-

AMBARI-17247 : Populate audit to solr / hdfs properties for Atlas


Bugs: AMBARI-17247
https://issues.apache.org/jira/browse/AMBARI-17247


Repository: ambari


Description (updated)
---

**Issue :**
If Ranger Atlas Plugin is enabled and audit to solr or hdfs are enabled from 
Ranger then, need to populate solr / hdfs audit configurations in atlas configs.


Diffs (updated)
-

  
ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/configuration/ranger-atlas-plugin-properties.xml
 f3bdc2a 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
614f0b1 

Diff: https://reviews.apache.org/r/48726/diff/


Testing
---

Verified recommendations for audit to solr as well as hdfs for Atlas (if 
enabled from Ranger configs)


Thanks,

Gautam Borad



Re: Review Request 48702: Add ability to set GET request directives

2016-06-15 Thread Di Li

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48702/#review137721
---


Ship it!




Ship It!

- Di Li


On June 14, 2016, 11:12 p.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48702/
> ---
> 
> (Updated June 14, 2016, 11:12 p.m.)
> 
> 
> Review request for Ambari, Ajit Kumar, Di Li, Jonathan Hurley, and Nate Cole.
> 
> 
> Bugs: AMBARI-17224
> https://issues.apache.org/jira/browse/AMBARI-17224
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add ability to set GET request directives so that directives may be specified 
> in a GET requests. For example, to limit or increase the amount of work 
> needed to be performed internally to generate the data being returned.
> 
> Most data in GET requests are retrieved but filtered from the response based 
> on the predicate. This means that any data the requires significant 
> processing to calculate will be calculated regardless of the requested 
> fields. By using a directive this can be controlled.
> 
> Note: It may be possible the use of a PropertyProvider to control this 
> instead, but that may not be desired in all cases.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/handlers/ReadHandler.java
>  95e45d6 
>   ambari-server/src/main/java/org/apache/ambari/server/api/query/Query.java 
> fff842a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/query/QueryImpl.java 
> 18fd94b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/BaseResourceDefinition.java
>  dcc5217 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/ResourceDefinition.java
>  8df1a02 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/SimpleResourceDefinition.java
>  394e295 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/UpgradeResourceDefinition.java
>  bf7860b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/BaseRequest.java
>  0ebeb19 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/RequestFactory.java
>  19fee8e 
>   
> ambari-server/src/test/java/org/apache/ambari/server/api/handlers/ReadHandlerTest.java
>  5cb601e 
>   
> ambari-server/src/test/java/org/apache/ambari/server/api/resources/BaseResourceDefinitionTest.java
>  f174cb6 
>   
> ambari-server/src/test/java/org/apache/ambari/server/api/resources/SimpleResourceDefinitionTest.java
>  6169ee7 
>   
> ambari-server/src/test/java/org/apache/ambari/server/api/services/RequestFactoryTest.java
>  11568c9 
>   
> ambari-server/src/test/java/org/apache/ambari/server/audit/request/creator/AuditEventCreatorTestHelper.java
>  2642418 
> 
> Diff: https://reviews.apache.org/r/48702/diff/
> 
> 
> Testing
> ---
> 
> # Local test results: 
> 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 1:25:36.424s
> [INFO] Finished at: Tue Jun 14 14:53:08 EDT 2016
> [INFO] Final Memory: 59M/1864M
> [INFO] 
> 
> 
> # Jenkins test results: PENDING
> 
> 
> Thanks,
> 
> Robert Levas
> 
>



Re: Review Request 48730: AMBARI-17250 : Use right principals for Hbase Master in Kerberos enabled Ranger Hbase Plugin

2016-06-15 Thread Jonathan Hurley

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48730/#review137741
---


Ship it!




Ship It!

- Jonathan Hurley


On June 15, 2016, 10:26 a.m., Gautam Borad wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48730/
> ---
> 
> (Updated June 15, 2016, 10:26 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Srimanth 
> Gunturi, and Velmurugan Periasamy.
> 
> 
> Bugs: AMBARI-17250
> https://issues.apache.org/jira/browse/AMBARI-17250
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> **Problem Statement :**
> Hbase test connection and resource lookup fails on secure kerberized cluster 
> due to invalid principal for Hbase Master. 
> Ambari configs are passing hbase_jaas_principal need to correct that.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py
>  76cefe7 
> 
> Diff: https://reviews.apache.org/r/48730/diff/
> 
> 
> Testing
> ---
> 
> Verified Restart and test connection for Hbase after enabling Ranger Hbase 
> Plugin in kerberos env. 
> Verified resource lookup for Hbase in Ranger (in Kerberos env)
> 
> 
> Thanks,
> 
> Gautam Borad
> 
>



Re: Review Request 48732: (Client) components that are dependencies of services in the stack definitions are always added to blueprint deployments

2016-06-15 Thread Robert Nettleton

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48732/#review137745
---


Ship it!




- Robert Nettleton


On June 15, 2016, 3:04 p.m., Laszlo Puskas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48732/
> ---
> 
> (Updated June 15, 2016, 3:04 p.m.)
> 
> 
> Review request for Ambari, Robert Nettleton, Sandor Magyari, and Sebastian 
> Toader.
> 
> 
> Bugs: AMBARI-17251
> https://issues.apache.org/jira/browse/AMBARI-17251
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Problem:
> Component dependencies from stack definitions are automatically added to 
> blueprint deployments even the services components belong to are not listed 
> in the blueprint.
> eg.: SPARK_CLIENT/TEZ_CLIENT components that are listed as dependencies of 
> the ATS and RESOURCEMANAGER in the stack definitions are always added to 
> blueprints.
> 
> Solution:
> Component dependencies defined in stack definitions are only added to 
> blueprints if services belonging to the listed components are part of the 
> blueprint
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintValidatorImpl.java
>  432c6f8 
>   
> ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintValidatorImplTest.java
>  f78d86d 
> 
> Diff: https://reviews.apache.org/r/48732/diff/
> 
> 
> Testing
> ---
> 
> Unit tests running;
> Manually tested on local env.
> 
> 
> Thanks,
> 
> Laszlo Puskas
> 
>



Re: Review Request 48727: Log Search default log levels can not be altered

2016-06-15 Thread Robert Nettleton

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48727/#review137746
---


Ship it!




Ship It!

- Robert Nettleton


On June 15, 2016, 12:46 p.m., Miklos Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48727/
> ---
> 
> (Updated June 15, 2016, 12:46 p.m.)
> 
> 
> Review request for Ambari, Don Bosco Durai, Dharmesh Makwana, Oliver Szabo, 
> Robert Nettleton, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17254
> https://issues.apache.org/jira/browse/AMBARI-17254
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> The default log levels (log levels to include) can not be altered, forever 
> the ones set during the installation (more precisely: the one which is the 
> value of the property during the first run of the portal) would be in effect.
> 
> Currently the portal checks if there is a configuration, and does nothing if 
> there is. After this change it will remove any existing configuration upon 
> starting, and replace it with a newly generated configuration.
> 
> 
> Diffs
> -
> 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/SolrDaoBase.java
>  147e148 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/UserConfigSolrDao.java
>  edf1dcc 
> 
> Diff: https://reviews.apache.org/r/48727/diff/
> 
> 
> Testing
> ---
> 
> Tested on local cluster.
> 
> 
> Thanks,
> 
> Miklos Gergely
> 
>



Re: Review Request 48722: Reduce the idle time before first command from next stage is executed on a host

2016-06-15 Thread Sebastian Toader

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48722/
---

(Updated June 15, 2016, 4:55 p.m.)


Review request for Ambari, Andrew Onischuk, Laszlo Puskas, Robert Levas, Sandor 
Magyari, and Sumit Mohanty.


Changes
---

Fix failing unit tests


Bugs: AMBARI-17248
https://issues.apache.org/jira/browse/AMBARI-17248


Repository: ambari


Description
---

Commands to be executed by ambari-agents are being sent down by the server in 
the response message to agent heartbeat messages. 
The server processes the received heartbeat, it checks if there are next 
commands scheduled to be executed by ambari-agent and adds those to the 
heartbeat response for the ambari-agent.
The server organises the commands that can be executed in parallel into stages. 
Ambari server ensures that only the commands of a single stage is scheduled to 
be executed by the agent and starts scheduling the commands of the next stage 
only after all commands of current stage has finished successfully.
The processing of command status received with the heartbeat message happens 
asynchronously to heartbeat response in HeartBeatProcessor and ActionScheduler 
creation thus when the heartbeat response is created the commands for the next 
stage are not scheduled yet. This means that the next commands will be sent to 
agent only with the next heartbeat.
Agents currently sends a heartbeat to the server on command a completion or at 
a timeout = self.netutil.HEARTBEAT_IDDLE_INTERVAL_SEC – 
self.netutil.MINIMUM_INTERVAL_BETWEEN_HEARTBEATS interval which is ~10 seconds 
if there are no commands to be executed.
This means that when the server receives a heartbeat triggered by the 
completion of the last command from the current stage the server will send the 
commands for the next stage only 10 seconds later when the next heartbeat is 
received. This leads to agents spending considerable amount of time idle when 
there are multiple stages to be executed.
Agents should heartbeat at a higher rate while there are still pending stages 
to be executed.


Diffs (updated)
-

  ambari-agent/src/main/python/ambari_agent/Controller.py e981a76 
  ambari-agent/src/main/python/ambari_agent/NetUtil.py 80bf3ae 
  ambari-agent/src/test/python/ambari_agent/TestNetUtil.py d72e319 
  ambari-agent/src/test/python/ambari_agent/examples/ControllerTester.py 
8103872 
  
ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java
 35a37e3 
  
ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatResponse.java
 1ab7ae9 
  ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java 
ac0ddd2 
  ambari-server/src/main/java/org/apache/ambari/server/state/Clusters.java 
bd9de13 
  
ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
 3d2388e 
  
ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClustersImpl.java
 c26e1e9 
  
ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClusterImplTest.java
 627ade9 

Diff: https://reviews.apache.org/r/48722/diff/


Testing (updated)
---

Manual testing.

Unit tests in succeeded.


Thanks,

Sebastian Toader



Review Request 48732: (Client) components that are dependencies of services in the stack definitions are always added to blueprint deployments

2016-06-15 Thread Laszlo Puskas

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48732/
---

Review request for Ambari, Robert Nettleton, Sandor Magyari, and Sebastian 
Toader.


Bugs: AMBARI-17251
https://issues.apache.org/jira/browse/AMBARI-17251


Repository: ambari


Description
---

Problem:
Component dependencies from stack definitions are automatically added to 
blueprint deployments even the services components belong to are not listed in 
the blueprint.
eg.: SPARK_CLIENT/TEZ_CLIENT components that are listed as dependencies of the 
ATS and RESOURCEMANAGER in the stack definitions are always added to blueprints.

Solution:
Component dependencies defined in stack definitions are only added to 
blueprints if services belonging to the listed components are part of the 
blueprint


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintValidatorImpl.java
 432c6f8 
  
ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintValidatorImplTest.java
 f78d86d 

Diff: https://reviews.apache.org/r/48732/diff/


Testing
---

Unit tests running;
Manually tested on local env.


Thanks,

Laszlo Puskas



Re: Review Request 47656: AMBARI-12885 - Dynamic stack extensions - install and upgrade support for custom services

2016-06-15 Thread Tim Thorpe

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47656/
---

(Updated June 15, 2016, 2:08 p.m.)


Review request for Ambari, Alexander Denissov, Alejandro Fernandez, bhuvnesh 
chaudhary, Jayush Luniya, Oleksandr Diachenko, Sumit Mohanty, Srimanth Gunturi, 
and Yusaku Sako.


Changes
---

Change unq_ to UQ_ in sql files.


Bugs: AMBARI-12885
https://issues.apache.org/jira/browse/AMBARI-12885


Repository: ambari


Description
---

The purpose of this proposal is to facilitate adding custom services to an 
existing stack. Ideally this would support adding and upgrading custom services 
separately from the core services defined in the stack. In particular we are 
looking at custom services that need to support several different stacks 
(different distributions of Ambari). The release cycle of the custom services 
may be different from that of the core stack; that is, a custom service may be 
upgraded at a different rate than the core distribution itself and may be 
upgraded multiple times within the lifespan of a single release of the core 
distribution.

One possible approach to handling this would be dynamically extending a stack 
(after install time). It would be best to extend the stack in packages where a 
stack extension package can have one or more custom services.


Diffs (updated)
-

  ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py 
fc1b72a 
  ambari-agent/src/main/python/ambari_agent/FileCache.py b7c5dee 
  ambari-server/conf/unix/ambari.properties a88a025 
  ambari-server/src/main/assemblies/server.xml 1560d8d 
  
ambari-server/src/main/java/org/apache/ambari/server/api/resources/ExtensionLinkResourceDefinition.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/api/resources/ExtensionResourceDefinition.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/api/resources/ExtensionVersionResourceDefinition.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/api/resources/ResourceInstanceFactoryImpl.java
 cf7c391 
  
ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java
 f0928cf 
  
ambari-server/src/main/java/org/apache/ambari/server/api/services/ExtensionLinksService.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/api/services/ExtensionsService.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/api/services/StacksService.java
 557ce98 
  
ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
 2b7fca0 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementController.java
 b488af3 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
 ba93d25 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/ExtensionLinkRequest.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/ExtensionLinkResponse.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/ExtensionRequest.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/ExtensionResponse.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/ExtensionVersionRequest.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/ExtensionVersionResponse.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractControllerResourceProvider.java
 3721113 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/Extension.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ExtensionLinkResourceProvider.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ExtensionResourceProvider.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ExtensionVersionResourceProvider.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/spi/Resource.java
 99e4ccd 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/dao/ExtensionDAO.java 
PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/dao/ExtensionLinkDAO.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/ExtensionEntity.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/ExtensionLinkEntity.java
 PRE-CREATION 
  ambari-server/src/main/java/org/apache/ambari/server/stack/BaseModule.java 
ef2438f 
  
ambari-server/src/main/java/org/apache/ambari/server/stack/CommonServiceDirectory.java
 cbbdb91 
  
ambari-server/src/main/java/org/apache/ambari/server/stack/ComponentModule.java 

Re: Review Request 48685: AMBARI-17218 Show message of Audit to DB Removal during upgrade for Ranger

2016-06-15 Thread Mugdha Varadkar

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48685/
---

(Updated June 15, 2016, 2:18 p.m.)


Review request for Ambari, Gautam Borad, Jonathan Hurley, Srimanth Gunturi, and 
Velmurugan Periasamy.


Changes
---

Handle Jonathan Hurley comments


Bugs: AMBARI-17218
https://issues.apache.org/jira/browse/AMBARI-17218


Repository: ambari


Description
---

Show Warning message before upgrade to stack 2.5


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/checks/CheckDescription.java
 850b58a 
  
ambari-server/src/main/java/org/apache/ambari/server/checks/RangerAuditDbCheck.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/checks/UpgradeCheckGroup.java
 34d016e 
  
ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml
 464c444 
  ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
dd04b64 
  
ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml
 27461e8 
  ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
ad1bc34 
  
ambari-server/src/test/java/org/apache/ambari/server/checks/RangerAuditDbCheckTest.java
 PRE-CREATION 

Diff: https://reviews.apache.org/r/48685/diff/


Testing
---

Tested upgrade from 2.4 to 2.5


---
 T E S T S
---
Picked up _JAVA_OPTIONS: -Xmx2048m -XX:MaxPermSize=512m -Djava.awt.headless=true
Running org.apache.ambari.server.checks.RangerAuditDbCheckTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.743 sec - in 
org.apache.ambari.server.checks.RangerAuditDbCheckTest

Results :

Tests run: 2, Failures: 0, Errors: 0, Skipped: 0


Thanks,

Mugdha Varadkar



Re: Review Request 48685: AMBARI-17218 Show message of Audit to DB Removal during upgrade for Ranger

2016-06-15 Thread Mugdha Varadkar


> On June 15, 2016, 1:10 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/checks/UpgradeCheckGroup.java,
> >  line 85
> > 
> >
> > Let's bump this up to something like 100.0f for now (they should only 
> > appear after all other issues are resolved)

Updated in latest patch


> On June 15, 2016, 1:10 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/checks/CheckDescription.java,
> >  lines 282-283
> > 
> >
> > Since we're keeping this, let's also revise it to flow a little easier:
> > 
> > ```
> > After upgrading, Ranger will no longer support the Audit to Database 
> > feature. Instead, Ranger will audit to Solr. To migrate the existing logs 
> > to Solr, run the following migration script: 
> > ```

Right now, wiki page link / script is not available : once we have any one of 
these ... will update the message. All the needed steps to perform migration 
are ready.


- Mugdha


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48685/#review137725
---


On June 15, 2016, 2:18 p.m., Mugdha Varadkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48685/
> ---
> 
> (Updated June 15, 2016, 2:18 p.m.)
> 
> 
> Review request for Ambari, Gautam Borad, Jonathan Hurley, Srimanth Gunturi, 
> and Velmurugan Periasamy.
> 
> 
> Bugs: AMBARI-17218
> https://issues.apache.org/jira/browse/AMBARI-17218
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Show Warning message before upgrade to stack 2.5
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/CheckDescription.java
>  850b58a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/RangerAuditDbCheck.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/UpgradeCheckGroup.java
>  34d016e 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml
>  464c444 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> dd04b64 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml
>  27461e8 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> ad1bc34 
>   
> ambari-server/src/test/java/org/apache/ambari/server/checks/RangerAuditDbCheckTest.java
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/48685/diff/
> 
> 
> Testing
> ---
> 
> Tested upgrade from 2.4 to 2.5
> 
> 
> ---
>  T E S T S
> ---
> Picked up _JAVA_OPTIONS: -Xmx2048m -XX:MaxPermSize=512m 
> -Djava.awt.headless=true
> Running org.apache.ambari.server.checks.RangerAuditDbCheckTest
> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.743 sec - 
> in org.apache.ambari.server.checks.RangerAuditDbCheckTest
> 
> Results :
> 
> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0
> 
> 
> Thanks,
> 
> Mugdha Varadkar
> 
>



Re: Review Request 48651: Add unit tests for Spark2 service definition

2016-06-15 Thread Jayush Luniya

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48651/#review137765
---



Can you rebase patch to latest in trunk and submit the patch

git apply ~/Downloads/AMBARI-16864.patch
/Users/jluniya/Downloads/AMBARI-16864.patch:1114: trailing whitespace.
"spark-javaopts-properties": {},
/Users/jluniya/Downloads/AMBARI-16864.patch:1125: trailing whitespace.
},
/Users/jluniya/Downloads/AMBARI-16864.patch:1127: trailing whitespace.
"service_package_folder": "common-services/SPARK/1.2.1/package",
/Users/jluniya/Downloads/AMBARI-16864.patch:1128: trailing whitespace.
"script": "scripts/job_history_server.py",
/Users/jluniya/Downloads/AMBARI-16864.patch:1129: trailing whitespace.
"hooks_folder": "HDP/2.0.6/hooks",
error: patch failed: 
ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/params.py:156
error: 
ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/params.py:
 patch does not apply

- Jayush Luniya


On June 13, 2016, 4:28 p.m., Saisai Shao wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48651/
> ---
> 
> (Updated June 13, 2016, 4:28 p.m.)
> 
> 
> Review request for Ambari and Jayush Luniya.
> 
> 
> Bugs: AMBARI-16864
> https://issues.apache.org/jira/browse/AMBARI-16864
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add unit tests for Spark2 service definition
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/constants.py
>  7e85115 
>   
> ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/params.py
>  6925ab5 
>   
> ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/spark_client.py
>  2c19b88 
>   
> ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/spark_service.py
>  c2385df 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_features.json
>  8ad53da 
>   ambari-server/src/test/python/stacks/2.5/SPARK2/test_job_history_server.py 
> PRE-CREATION 
>   ambari-server/src/test/python/stacks/2.5/SPARK2/test_spark_client.py 
> PRE-CREATION 
>   ambari-server/src/test/python/stacks/2.5/SPARK2/test_spark_service_check.py 
> PRE-CREATION 
>   ambari-server/src/test/python/stacks/2.5/SPARK2/test_spark_thrift_server.py 
> PRE-CREATION 
>   ambari-server/src/test/python/stacks/2.5/configs/secured.json PRE-CREATION 
>   
> ambari-server/src/test/python/stacks/2.5/configs/spark2-job-history-server.json
>  PRE-CREATION 
>   ambari-server/src/test/python/stacks/2.5/configs/spark2_default.json 
> PRE-CREATION 
>   ambari-server/src/test/python/stacks/2.5/configs/spark2_thriftserver.json 
> PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/48651/diff/
> 
> 
> Testing
> ---
> 
> Local unit test is done.
> 
> 
> Thanks,
> 
> Saisai Shao
> 
>



Re: Review Request 47858: Cache service advisors when stack advisor is loaded

2016-06-15 Thread Jayush Luniya


> On June 1, 2016, 8:55 p.m., Jayush Luniya wrote:
> > Ship It!

Lav, can you close the review if the patch is already committed?


- Jayush


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47858/#review135854
---


On May 27, 2016, 7:11 p.m., Lav Jain wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47858/
> ---
> 
> (Updated May 27, 2016, 7:11 p.m.)
> 
> 
> Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, Jayush 
> Luniya, Matt, and Tim Thorpe.
> 
> 
> Bugs: AMBARI-16665
> https://issues.apache.org/jira/browse/AMBARI-16665
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> With the current implementation, service advisors for the same service are 
> loaded multiple times in the stack advisor.
> 
> The service advisors should be cached when the stack advisor is loaded and 
> used where necessary.
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/stacks/stack_advisor.py 63885d2 
>   ambari-server/src/test/python/stacks/2.3/HAWQ/test_service_advisor.py 
> 1d6a85c 
> 
> Diff: https://reviews.apache.org/r/47858/diff/
> 
> 
> Testing
> ---
> 
> Lavs-MacBook-Pro:common ljain$ python -m discover -v
> test_createComponentLayoutRecommendations_hawq_1_Host 
> (test_stack_advisor.TestHDP23StackAdvisor) ... ServiceAdvisor implementation 
> for service HAWQ was loaded
> ok
> test_createComponentLayoutRecommendations_hawq_3_Hosts 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test that HAWQSTANDBY is recommended on a 3-node cluster ... ServiceAdvisor 
> implementation for service HAWQ was loaded
> ok
> test_createComponentLayoutRecommendations_hawqsegment_add_service_wizard_already_installed
>  (test_stack_advisor.TestHDP23StackAdvisor)
> Test that HAWQSEGMENT does not get recommended during Add Service Wizard, 
> when HAWQ has already been installed ... ServiceAdvisor implementation for 
> service HAWQ was loaded
> ok
> test_createComponentLayoutRecommendations_hawqsegment_add_service_wizard_to_be_installed
>  (test_stack_advisor.TestHDP23StackAdvisor)
> Test that HAWQSEGMENT gets recommended correctly during Add Service Wizard, 
> when HAWQ is selected for installation ... ServiceAdvisor implementation for 
> service HAWQ was loaded
> ok
> test_createComponentLayoutRecommendations_hawqsegment_cluster_install 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test that HAWQSEGMENT gets recommended correctly during Cluster Install 
> Wizard, when HAWQ is selected for installation ... ServiceAdvisor 
> implementation for service HAWQ was loaded
> ok
> test_createComponentLayoutRecommendations_no_hawq_3_Hosts 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test no failures when there are no HAWQ components ... ok
> test_createComponentLayoutRecommendations_pxf_add_service_wizard_already_installed
>  (test_stack_advisor.TestHDP23StackAdvisor)
> Test that PXF does not get recommended during Add Service Wizard, when PXF 
> has already been installed ... ServiceAdvisor implementation for service PXF 
> was loaded
> ok
> test_createComponentLayoutRecommendations_pxf_add_service_wizard_to_be_installed
>  (test_stack_advisor.TestHDP23StackAdvisor)
> Test that PXF gets recommended correctly during Add Service Wizard, when PXF 
> is selected for installation ... ServiceAdvisor implementation for service 
> PXF was loaded
> ok
> test_createComponentLayoutRecommendations_pxf_cluster_install 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test that PXF gets recommended correctly during Cluster Install Wizard, when 
> PXF is selected for installation ... ServiceAdvisor implementation for 
> service PXF was loaded
> ok
> test_getComponentLayoutValidations_hawq_3_Hosts 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test layout validations for HAWQ components on a 3-node cluster ... 
> ServiceAdvisor implementation for service HAWQ was loaded
> ok
> test_getComponentLayoutValidations_hawqsegment_not_co_located_with_datanode 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test validation warning for HAWQ segment not colocated with DATANODE ... 
> ServiceAdvisor implementation for service HAWQ was loaded
> ok
> test_getComponentLayoutValidations_nohawq_3_Hosts 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test no failures when there are no HAWQ components on a 3-node cluster ... ok
> test_getComponentLayoutValidations_pxf_co_located_with_nn_and_dn 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test NO warning is generated when PXF is co-located with NAMENODE and 
> DATANODE ... ServiceAdvisor implementation for service PXF was loaded
> ok
> test_getComponentLayoutValidations_pxf_not_co_located_with_dn 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test warning 

Re: Review Request 48636: Zeppelin Views are not working with Custom and Remote cluster view configuration

2016-06-15 Thread Jayush Luniya

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48636/#review137769
---




contrib/views/zeppelin/src/main/resources/view.xml (line 35)


fake?


- Jayush Luniya


On June 13, 2016, 1:22 p.m., Renjith Kamath wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48636/
> ---
> 
> (Updated June 13, 2016, 1:22 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, DIPAYAN BHOWMICK, Gaurav 
> Nagar, Jayush Luniya, Pallav Kulshreshtha, Rohit Choudhary, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17191
> https://issues.apache.org/jira/browse/AMBARI-17191
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Two patches attached in the jira issue
> AMBARI-17191_branch-2.4_v1.patch
> AMBARI-17191_trunk_v1.patch
> 
> 
> - fix zeppelin view
> 
> 
> Diffs
> -
> 
>   
> contrib/views/zeppelin/src/main/java/org/apache/ambari/view/zeppelin/ZeppelinServiceCheck.java
>  e80f884 
>   
> contrib/views/zeppelin/src/main/java/org/apache/ambari/view/zeppelin/ZeppelinServlet.java
>  f497599 
>   contrib/views/zeppelin/src/main/resources/WEB-INF/index.jsp 493473e 
>   contrib/views/zeppelin/src/main/resources/WEB-INF/web.xml 2ca5664 
>   contrib/views/zeppelin/src/main/resources/view.xml 3c5c5cf 
> 
> Diff: https://reviews.apache.org/r/48636/diff/
> 
> 
> Testing
> ---
> 
> Manually tested on CentOS
> 
> 
> Thanks,
> 
> Renjith Kamath
> 
>



Re: Review Request 48335: Zeppelin service: Update default zeppelin_pid_dir to /var/run/zeppelin

2016-06-15 Thread Jayush Luniya

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48335/#review137770
---


Ship it!




Ship It!

- Jayush Luniya


On June 7, 2016, 11:45 a.m., Renjith Kamath wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48335/
> ---
> 
> (Updated June 7, 2016, 11:45 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, DIPAYAN BHOWMICK, Jayush 
> Luniya, Pallav Kulshreshtha, Rohit Choudhary, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17088
> https://issues.apache.org/jira/browse/AMBARI-17088
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> update dir name from zeppelin-notebook to zeppelin
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/configuration/zeppelin-env.xml
>  efd31f1 
> 
> Diff: https://reviews.apache.org/r/48335/diff/
> 
> 
> Testing
> ---
> 
> manually tested on CentOS
> 
> 
> Thanks,
> 
> Renjith Kamath
> 
>



Re: Review Request 48651: Add unit tests for Spark2 service definition

2016-06-15 Thread Saisai Shao


> On June 15, 2016, 4:47 p.m., Jayush Luniya wrote:
> > Can you rebase patch to latest in trunk and submit the patch
> > 
> > git apply ~/Downloads/AMBARI-16864.patch
> > /Users/jluniya/Downloads/AMBARI-16864.patch:1114: trailing whitespace.
> > "spark-javaopts-properties": {},
> > /Users/jluniya/Downloads/AMBARI-16864.patch:1125: trailing whitespace.
> > },
> > /Users/jluniya/Downloads/AMBARI-16864.patch:1127: trailing whitespace.
> > "service_package_folder": "common-services/SPARK/1.2.1/package",
> > /Users/jluniya/Downloads/AMBARI-16864.patch:1128: trailing whitespace.
> > "script": "scripts/job_history_server.py",
> > /Users/jluniya/Downloads/AMBARI-16864.patch:1129: trailing whitespace.
> > "hooks_folder": "HDP/2.0.6/hooks",
> > error: patch failed: 
> > ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/params.py:156
> > error: 
> > ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/params.py:
> >  patch does not apply

Sure, I will.


- Saisai


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48651/#review137765
---


On June 13, 2016, 4:28 p.m., Saisai Shao wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48651/
> ---
> 
> (Updated June 13, 2016, 4:28 p.m.)
> 
> 
> Review request for Ambari and Jayush Luniya.
> 
> 
> Bugs: AMBARI-16864
> https://issues.apache.org/jira/browse/AMBARI-16864
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add unit tests for Spark2 service definition
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/constants.py
>  7e85115 
>   
> ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/params.py
>  6925ab5 
>   
> ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/spark_client.py
>  2c19b88 
>   
> ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/spark_service.py
>  c2385df 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_features.json
>  8ad53da 
>   ambari-server/src/test/python/stacks/2.5/SPARK2/test_job_history_server.py 
> PRE-CREATION 
>   ambari-server/src/test/python/stacks/2.5/SPARK2/test_spark_client.py 
> PRE-CREATION 
>   ambari-server/src/test/python/stacks/2.5/SPARK2/test_spark_service_check.py 
> PRE-CREATION 
>   ambari-server/src/test/python/stacks/2.5/SPARK2/test_spark_thrift_server.py 
> PRE-CREATION 
>   ambari-server/src/test/python/stacks/2.5/configs/secured.json PRE-CREATION 
>   
> ambari-server/src/test/python/stacks/2.5/configs/spark2-job-history-server.json
>  PRE-CREATION 
>   ambari-server/src/test/python/stacks/2.5/configs/spark2_default.json 
> PRE-CREATION 
>   ambari-server/src/test/python/stacks/2.5/configs/spark2_thriftserver.json 
> PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/48651/diff/
> 
> 
> Testing
> ---
> 
> Local unit test is done.
> 
> 
> Thanks,
> 
> Saisai Shao
> 
>



Re: Review Request 48308: [AMBARI-17078] Make Spark2-ThriftServer and Livy Server as optional by default

2016-06-15 Thread Jayush Luniya

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48308/#review137771
---


Ship it!




Ship It!

- Jayush Luniya


On June 14, 2016, 4:10 a.m., Jeff Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48308/
> ---
> 
> (Updated June 14, 2016, 4:10 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17078
> https://issues.apache.org/jira/browse/AMBARI-17078
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Spark2-ThriftServer and Livy Server are not being selected by default, but 
> should be unselected by default.
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
> db2986b 
> 
> Diff: https://reviews.apache.org/r/48308/diff/
> 
> 
> Testing
> ---
> 
> Manullay verified
> 
> 
> Thanks,
> 
> Jeff Zhang
> 
>



Review Request 48735: Storm service check failed after Ambari upgrade

2016-06-15 Thread Dmitro Lisnichenko

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48735/
---

Review request for Ambari, Jonathan Hurley and Nate Cole.


Bugs: AMBARI-17258
https://issues.apache.org/jira/browse/AMBARI-17258


Repository: ambari


Description
---

ambari-server-2.4.0.0-665.x86_64
ambari-server --hash

*Steps*
# Deploy HDP-2.3.2.0 cluster with Ambari 2.1.2 (secure, non-HA cluster)
# Upgrade Ambari to 2.4.0.0-665
# Stop and Start all services
# Run service check for Storm

*Result*
Failure as below:
{code}
stderr:   /var/lib/ambari-agent/data/errors-729.txt

Traceback (most recent call last):
File 
"/var/lib/ambari-agent/cache/common-services/STORM/0.9.1/package/scripts/service_check.py",
 line 79, in 
ServiceCheck().execute()
File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
 line 257, in execute
method(env)
File 
"/var/lib/ambari-agent/cache/common-services/STORM/0.9.1/package/scripts/service_check.py",
 line 70, in service_check
user=params.storm_user
File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 
155, in __init__
self.env.run()
File 
"/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
line 160, in run
self.run_action(resource, action)
File 
"/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
line 124, in run_action
provider_action()
File 
"/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py",
 line 273, in action_run
tries=self.resource.tries, try_sleep=self.resource.try_sleep)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
70, in inner
result = function(command, **kwargs)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
92, in checked_call
tries=tries, try_sleep=try_sleep)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
140, in _call_wrapper
result = _call(command, **kwargs_copy)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
293, in _call
raise Fail(err_msg)
resource_management.core.exceptions.Fail: Execution of 'storm jar 
/tmp/wordCount.jar storm.starter.WordCountTopology 
WordCountid16ac396c_date241516' returned 1. 951  [main] INFO  b.s.u.Utils - 
Using defaults.yaml from resources
1088 [main] INFO  b.s.u.Utils - Using storm.yaml from resources
1184 [main] INFO  b.s.u.Utils - Using defaults.yaml from resources
1240 [main] INFO  b.s.u.Utils - Using storm.yaml from resources
1285 [main] INFO  b.s.StormSubmitter - Generated ZooKeeper secret payload for 
MD5-digest: -7463484230184273671:-7128399429695564111
1287 [main] INFO  b.s.s.a.AuthUtils - Got AutoCreds []
1292 [main] WARN  b.s.u.NimbusClient - Using deprecated config nimbus.host for 
backward compatibility. Please update your storm.yaml so it only has config 
nimbus.seeds
1352 [main] INFO  b.s.u.StormBoundedExponentialBackoffRetry - The 
baseSleepTimeMs [2000] the maxSleepTimeMs [6] the maxRetries [5]
1615 [main] INFO  o.a.s.z.Login - successfully logged in.
1650 [main] ERROR o.a.t.t.TSaslTransport - SASL negotiation failure
javax.security.sasl.SaslException: GSS initiate failed
at 
com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:212)
 ~[?:1.7.0_67]
at 
org.apache.thrift7.transport.TSaslClientTransport.handleSaslStartMessage(TSaslClientTransport.java:94)
 ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at org.apache.thrift7.transport.TSaslTransport.open(TSaslTransport.java:271) 
[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at 
org.apache.thrift7.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
 [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at 
backtype.storm.security.auth.kerberos.KerberosSaslTransportPlugin$1.run(KerberosSaslTransportPlugin.java:145)
 [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at 
backtype.storm.security.auth.kerberos.KerberosSaslTransportPlugin$1.run(KerberosSaslTransportPlugin.java:141)
 [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at java.security.AccessController.doPrivileged(Native Method) ~[?:1.7.0_67]
at javax.security.auth.Subject.doAs(Subject.java:415) [?:1.7.0_67]
at 
backtype.storm.security.auth.kerberos.KerberosSaslTransportPlugin.connect(KerberosSaslTransportPlugin.java:140)
 [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at 
backtype.storm.security.auth.TBackoffConnect.doConnectWithRetry(TBackoffConnect.java:48)
 [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at backtype.storm.security.auth.ThriftClient.reconnect(ThriftClient.java:103) 
[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at backtype.storm.security.auth.ThriftClient.(ThriftClient.java:72) 
[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at 

Review Request 48741: LogSearch Solr kerberos support

2016-06-15 Thread Oliver Szabo

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48741/
---

Review request for Ambari, Don Bosco Durai, Miklos Gergely, Robert Levas, 
Robert Nettleton, Sumit Mohanty, and Sebastian Toader.


Bugs: AMBARI-16736
https://issues.apache.org/jira/browse/AMBARI-16736


Repository: ambari


Description
---

Basuc logsearch solr kerberos support.


Diffs
-

  
ambari-common/src/main/python/resource_management/libraries/functions/solr_cloud_util.py
 e0a30ed 
  ambari-logsearch/ambari-logsearch-logfeeder/pom.xml bbe0bc9 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/OutputSolr.java
 b14c273 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/resources/logfeeder.properties
 076c09c 
  ambari-logsearch/ambari-logsearch-portal/pom.xml e83c75b 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/SolrDaoBase.java
 cd4ef93 
  
ambari-logsearch/ambari-logsearch-portal/src/main/resources/logsearch.properties
 37c8317 
  ambari-logsearch/ambari-logsearch-solr-client/pom.xml b634928 
  
ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/AmbariSolrCloudCLI.java
 a2da737 
  
ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/AmbariSolrCloudClient.java
 2805b0b 
  
ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/AmbariSolrCloudClientBuilder.java
 0813221 
  ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/alerts.json 
85762e0 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logfeeder-env.xml
 d27067c 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-env.xml
 3682e5d 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-solr-env.xml
 2420be0 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/kerberos.json 
6dd4aa7 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/logfeeder.py
 ce7f71c 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/logsearch.py
 2b5fdf7 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/logsearch_common.py
 d0ac389 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/logsearch_solr.py
 b55f3d6 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/params.py
 8a1449d 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logfeeder.py
 5d1ca85 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch.py
 368db03 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch_solr.py
 eac60db 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/status_params.py
 1efe605 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/templates/logfeeder.properties.j2
 6a52708 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/templates/logfeeder_jaas.conf.j2
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/templates/logsearch.properties.j2
 9ada5bf 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/templates/logsearch_jaas.conf.j2
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/templates/logsearch_solr_jaas.conf.j2
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/templates/zoo.cfg.j2
 1f3808b 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/logsearch-solr-env.sh.j2
 9f76d18 
  ambari-server/src/test/python/stacks/2.4/LOGSEARCH/test_solr.py 7cbbfc4 

Diff: https://reviews.apache.org/r/48741/diff/


Testing
---

ambari server python unit test output:

Total run:1061
Total errors:0
Total failures:0


Thanks,

Oliver Szabo



Re: Review Request 48609: AMBARI-17183: client.properties for Falcon should be configurable via Ambari

2016-06-15 Thread Venkat Ranganathan

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48609/
---

(Updated June 15, 2016, 10:51 a.m.)


Review request for Ambari.


Changes
---

Updated patch with suggestions


Bugs: AMBARI-17183
https://issues.apache.org/jira/browse/AMBARI-17183


Repository: ambari


Description
---

Currently falcon-client.properties is generated without functionality for user 
changes.   We should fix it for users of mirroring etc


Diffs (updated)
-

  
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon.py
 b5b3a34 
  
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/params_linux.py
 22fb691 
  
ambari-server/src/main/resources/stacks/HDP/2.1.GlusterFS/services/FALCON/package/scripts/falcon.py
 9a72af1 
  
ambari-server/src/main/resources/stacks/HDP/2.3/services/FALCON/configuration/falcon-client.properties.xml
 PRE-CREATION 

Diff: https://reviews.apache.org/r/48609/diff/


Testing
---


Thanks,

Venkat Ranganathan



Re: Review Request 48044: Provide context for hdp-select failures during ambari component install

2016-06-15 Thread Jayush Luniya


> On June 15, 2016, 5:51 p.m., Jayush Luniya wrote:
> > Ship It!

For 2.4 and trunk, we should not have any hdp-select and hdp hardcodings. 
Everything should be stack config driven.


- Jayush


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48044/#review137780
---


On June 13, 2016, 9:22 a.m., Laszlo Puskas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48044/
> ---
> 
> (Updated June 13, 2016, 9:22 a.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Jayush Luniya, Sandor Magyari, 
> Sumit Mohanty, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-16952
> https://issues.apache.org/jira/browse/AMBARI-16952
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In case the hdp-select command fails during a service / component 
> installation there's no contextual information about the cause of the failure.
> This issue is for logging information about the machine on which the 
> hdp-select command fails.
> This solution wraps hdp-select command calls in a try/catch block and logs 
> failure / hdp installation related information.
> 
> The patch only applies for 2.2-next versions.
> Based on the git logs the affected codebase has been refactored as part of 
> AMBARI-15329.
> (This addition should be ported to the newer versions isf applicable)
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/hdp_select.py
>  9a3201e 
> 
> Diff: https://reviews.apache.org/r/48044/diff/
> 
> 
> Testing
> ---
> 
> Unit tests passed.
> 
> 
> Thanks,
> 
> Laszlo Puskas
> 
>



Re: Review Request 48494: Implement config values trimming for deployment via blueprint

2016-06-15 Thread Dmytro Sen

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48494/
---

(Updated Июнь 15, 2016, 1:30 п.п.)


Review request for Ambari, Robert Nettleton, Sebastian Toader, and Vitalyi 
Brodetskyi.


Bugs: AMBARI-17146
https://issues.apache.org/jira/browse/AMBARI-17146


Repository: ambari


Description
---

Implement config values trimming for deployment via blueprint as we do in UI

  trimProperty: function (property) {
var displayType = Em.get(property, 'displayType');
var value = Em.get(property, 'value');
var name = Em.get(property, 'name');
var rez;
switch (displayType) {
  case 'directories':
  case 'directory':
rez = value.replace(/,/g, ' ').trim().split(/\s+/g).join(',');
break;
  case 'host':
rez = value.trim();
break;
  case 'password':
break;
  default:
if (name == 'javax.jdo.option.ConnectionURL' || name == 
'oozie.service.JPAService.jdbc.url') {
  rez = value.trim();
}
rez = (typeof value == 'string') ? value.replace(/(\s+$)/g, '') : value;
}
return ((rez == '') || (rez == undefined)) ? value : rez;
  },


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
 9094698 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/DefaultTrimmingStrategy.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/DeleteSpacesAtTheEndTrimmingStrategy.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/DirectoriesTrimmingStrategy.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/PasswordTrimmingStrategy.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/PropertyValueTrimmingStrategyDefiner.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/Stack.java
 ad8d4f9 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/TrimmingStrategy.java
 PRE-CREATION 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
 cda8fb8 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/StackTest.java
 e70af3e 

Diff: https://reviews.apache.org/r/48494/diff/


Testing
---

Unit tests and manual tests passed


Thanks,

Dmytro Sen



Re: Review Request 48494: Implement config values trimming for deployment via blueprint

2016-06-15 Thread Daniel Gergely

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48494/#review137731
---


Fix it, then Ship it!





ambari-server/src/main/java/org/apache/ambari/server/controller/internal/DeleteSpacesAtTheEndTrimmingStrategy.java
 (line 24)


Please swap variable and constant.


- Daniel Gergely


On jún. 15, 2016, 1:30 du, Dmytro Sen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48494/
> ---
> 
> (Updated jún. 15, 2016, 1:30 du)
> 
> 
> Review request for Ambari, Robert Nettleton, Sebastian Toader, and Vitalyi 
> Brodetskyi.
> 
> 
> Bugs: AMBARI-17146
> https://issues.apache.org/jira/browse/AMBARI-17146
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Implement config values trimming for deployment via blueprint as we do in UI
> 
>   trimProperty: function (property) {
> var displayType = Em.get(property, 'displayType');
> var value = Em.get(property, 'value');
> var name = Em.get(property, 'name');
> var rez;
> switch (displayType) {
>   case 'directories':
>   case 'directory':
> rez = value.replace(/,/g, ' ').trim().split(/\s+/g).join(',');
> break;
>   case 'host':
> rez = value.trim();
> break;
>   case 'password':
> break;
>   default:
> if (name == 'javax.jdo.option.ConnectionURL' || name == 
> 'oozie.service.JPAService.jdbc.url') {
>   rez = value.trim();
> }
> rez = (typeof value == 'string') ? value.replace(/(\s+$)/g, '') : 
> value;
> }
> return ((rez == '') || (rez == undefined)) ? value : rez;
>   },
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
>  9094698 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/DefaultTrimmingStrategy.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/DeleteSpacesAtTheEndTrimmingStrategy.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/DirectoriesTrimmingStrategy.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/PasswordTrimmingStrategy.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/PropertyValueTrimmingStrategyDefiner.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/Stack.java
>  ad8d4f9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/TrimmingStrategy.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
>  cda8fb8 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/StackTest.java
>  e70af3e 
> 
> Diff: https://reviews.apache.org/r/48494/diff/
> 
> 
> Testing
> ---
> 
> Unit tests and manual tests passed
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>



Re: Review Request 48494: Implement config values trimming for deployment via blueprint

2016-06-15 Thread Dmytro Sen

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48494/
---

(Updated Июнь 15, 2016, 2 п.п.)


Review request for Ambari, Robert Nettleton, Sebastian Toader, and Vitalyi 
Brodetskyi.


Bugs: AMBARI-17146
https://issues.apache.org/jira/browse/AMBARI-17146


Repository: ambari


Description
---

Implement config values trimming for deployment via blueprint as we do in UI

  trimProperty: function (property) {
var displayType = Em.get(property, 'displayType');
var value = Em.get(property, 'value');
var name = Em.get(property, 'name');
var rez;
switch (displayType) {
  case 'directories':
  case 'directory':
rez = value.replace(/,/g, ' ').trim().split(/\s+/g).join(',');
break;
  case 'host':
rez = value.trim();
break;
  case 'password':
break;
  default:
if (name == 'javax.jdo.option.ConnectionURL' || name == 
'oozie.service.JPAService.jdbc.url') {
  rez = value.trim();
}
rez = (typeof value == 'string') ? value.replace(/(\s+$)/g, '') : value;
}
return ((rez == '') || (rez == undefined)) ? value : rez;
  },


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
 9094698 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/DefaultTrimmingStrategy.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/DeleteSpacesAtTheEndTrimmingStrategy.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/DirectoriesTrimmingStrategy.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/PasswordTrimmingStrategy.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/PropertyValueTrimmingStrategyDefiner.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/Stack.java
 ad8d4f9 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/TrimmingStrategy.java
 PRE-CREATION 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
 cda8fb8 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/StackTest.java
 e70af3e 

Diff: https://reviews.apache.org/r/48494/diff/


Testing
---

Unit tests and manual tests passed


Thanks,

Dmytro Sen



Re: Review Request 48685: AMBARI-17218 Show message of Audit to DB Removal during upgrade for Ranger

2016-06-15 Thread Jonathan Hurley

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48685/#review137735
---


Ship it!




Ship It!

- Jonathan Hurley


On June 15, 2016, 10:18 a.m., Mugdha Varadkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48685/
> ---
> 
> (Updated June 15, 2016, 10:18 a.m.)
> 
> 
> Review request for Ambari, Gautam Borad, Jonathan Hurley, Srimanth Gunturi, 
> and Velmurugan Periasamy.
> 
> 
> Bugs: AMBARI-17218
> https://issues.apache.org/jira/browse/AMBARI-17218
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Show Warning message before upgrade to stack 2.5
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/CheckDescription.java
>  850b58a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/RangerAuditDbCheck.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/UpgradeCheckGroup.java
>  34d016e 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml
>  464c444 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> dd04b64 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml
>  27461e8 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> ad1bc34 
>   
> ambari-server/src/test/java/org/apache/ambari/server/checks/RangerAuditDbCheckTest.java
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/48685/diff/
> 
> 
> Testing
> ---
> 
> Tested upgrade from 2.4 to 2.5
> 
> 
> ---
>  T E S T S
> ---
> Picked up _JAVA_OPTIONS: -Xmx2048m -XX:MaxPermSize=512m 
> -Djava.awt.headless=true
> Running org.apache.ambari.server.checks.RangerAuditDbCheckTest
> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.743 sec - 
> in org.apache.ambari.server.checks.RangerAuditDbCheckTest
> 
> Results :
> 
> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0
> 
> 
> Thanks,
> 
> Mugdha Varadkar
> 
>



Re: Review Request 48730: AMBARI-17250 : Use right principals for Hbase Master in Kerberos enabled Ranger Hbase Plugin

2016-06-15 Thread Velmurugan Periasamy

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48730/#review137739
---


Ship it!




Ship It!

- Velmurugan Periasamy


On June 15, 2016, 2:26 p.m., Gautam Borad wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48730/
> ---
> 
> (Updated June 15, 2016, 2:26 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Srimanth 
> Gunturi, and Velmurugan Periasamy.
> 
> 
> Bugs: AMBARI-17250
> https://issues.apache.org/jira/browse/AMBARI-17250
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> **Problem Statement :**
> Hbase test connection and resource lookup fails on secure kerberized cluster 
> due to invalid principal for Hbase Master. 
> Ambari configs are passing hbase_jaas_principal need to correct that.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py
>  76cefe7 
> 
> Diff: https://reviews.apache.org/r/48730/diff/
> 
> 
> Testing
> ---
> 
> Verified Restart and test connection for Hbase after enabling Ranger Hbase 
> Plugin in kerberos env. 
> Verified resource lookup for Hbase in Ranger (in Kerberos env)
> 
> 
> Thanks,
> 
> Gautam Borad
> 
>



Re: Review Request 48607: AMBARI-17181: Add some of value-attributes to property files in AMBARI_METRICS

2016-06-15 Thread Aravindan Vijayan

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48607/#review137783
---



Masahiro, was this patch manually tested on a cluster?

- Aravindan Vijayan


On June 13, 2016, 12:18 a.m., Masahiro Tanaka wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48607/
> ---
> 
> (Updated June 13, 2016, 12:18 a.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-17181
> https://issues.apache.org/jira/browse/AMBARI-17181
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Some of the property files in AMBARI_METRICS lack value-attributes.
> It would be nice to have value-attributes on most of the properties. If so, 
> we can notice a careless mistakes in Ambari Server WebUI.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-grafana-env.xml
>  eaafc6b 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-grafana-ini.xml
>  da4599e 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-hbase-env.xml
>  b40923a 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-hbase-site.xml
>  f4e5fb2 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml
>  6484285 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-ssl-server.xml
>  6f9c6dc 
> 
> Diff: https://reviews.apache.org/r/48607/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Masahiro Tanaka
> 
>



Re: Review Request 48741: LogSearch Solr kerberos support

2016-06-15 Thread Robert Levas

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48741/#review137821
---


Ship it!





ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logfeeder-env.xml
 (line 135)


Is "none" an acceptable value here?  Maybe this should be ""?



ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logfeeder-env.xml
 (line 142)


Is "none" an acceptable value here?  Maybe this should be ""?



ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-env.xml
 (line 171)


Is "none" an acceptable value here?  Maybe this should be ""?



ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-env.xml
 (line 178)


Is "none" an acceptable value here?  Maybe this should be ""?



ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-solr-env.xml
 (line 192)


Is "none" an acceptable value here?  Maybe this should be ""?



ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-solr-env.xml
 (line 200)


Is "none" an acceptable value here?  Maybe this should be ""?



ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-solr-env.xml
 (line 208)


Is "none" an acceptable value here?  Maybe this should be ""?



ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-solr-env.xml
 (line 216)


Is "none" an acceptable value here?  Maybe this should be ""?


- Robert Levas


On June 15, 2016, 1:37 p.m., Oliver Szabo wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48741/
> ---
> 
> (Updated June 15, 2016, 1:37 p.m.)
> 
> 
> Review request for Ambari, Don Bosco Durai, Miklos Gergely, Robert Levas, 
> Robert Nettleton, Sumit Mohanty, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-16736
> https://issues.apache.org/jira/browse/AMBARI-16736
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Basuc logsearch solr kerberos support.
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/solr_cloud_util.py
>  e0a30ed 
>   ambari-logsearch/ambari-logsearch-logfeeder/pom.xml bbe0bc9 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/OutputSolr.java
>  b14c273 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/resources/logfeeder.properties
>  076c09c 
>   ambari-logsearch/ambari-logsearch-portal/pom.xml e83c75b 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/SolrDaoBase.java
>  cd4ef93 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/resources/logsearch.properties
>  37c8317 
>   ambari-logsearch/ambari-logsearch-solr-client/pom.xml b634928 
>   
> ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/AmbariSolrCloudCLI.java
>  a2da737 
>   
> ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/AmbariSolrCloudClient.java
>  2805b0b 
>   
> ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/AmbariSolrCloudClientBuilder.java
>  0813221 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/alerts.json 
> 85762e0 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logfeeder-env.xml
>  d27067c 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-env.xml
>  3682e5d 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-solr-env.xml
>  2420be0 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/kerberos.json
>  6dd4aa7 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/logfeeder.py
>  ce7f71c 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/logsearch.py
>  2b5fdf7 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/logsearch_common.py
>  d0ac389 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/logsearch_solr.py
>  b55f3d6 
>   
> 

Re: Review Request 47941: [AMBARI-16920] Follow up issue for Spark2 stack definition

2016-06-15 Thread Jayush Luniya

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47941/#review137788
---



@Jeff
Please rebase patch with latest in trunk. There are conflicts. Will commit once 
I have the latest patch.

- Jayush Luniya


On June 13, 2016, 10:45 a.m., Jeff Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47941/
> ---
> 
> (Updated June 13, 2016, 10:45 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-16920
> https://issues.apache.org/jira/browse/AMBARI-16920
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> This a followup ticket for add spark2 stack definition. There's serveral 
> issues:
> 1.  Spark2 thrift server can not started due to miss of 
> spark-thrift-fairscheduler.xml
> 2.  Miss of add spark2 cache file in copy_barball.py
> 3.  Miss the role_commnad_order of spark2
> 4.  conf_select is missign for spark2
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py
>  4eb0015 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py
>  286df8d 
>   
> ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/params.py
>  6925ab5 
>   
> ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/spark_service.py
>  c2385df 
>   ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 
> f6011b0 
> 
> Diff: https://reviews.apache.org/r/47941/diff/
> 
> 
> Testing
> ---
> 
> Manually verified.
> 
> 
> Thanks,
> 
> Jeff Zhang
> 
>



Re: Review Request 48395: AMBARI-17027: Metrics Collector API: Introduce basic series aggregation functions

2016-06-15 Thread Prajwal Rao

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48395/#review137790
---




ambari-metrics/ambari-metrics-grafana/ambari-metrics/queryCtrl.js (line 40)


There seems to be no way to disable Series Aggregation. We need to have a 
"none" option, and "sum" shouldn't be set as the default option.


- Prajwal Rao


On June 10, 2016, 12:50 a.m., Jungtaek Lim wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48395/
> ---
> 
> (Updated June 10, 2016, 12:50 a.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Dmytro Sen, Prajwal Rao, 
> Sriharsha Chintalapani, Sid Wagle, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-17027
> https://issues.apache.org/jira/browse/AMBARI-17027
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> AMS doesn't provide tag so metric is identified by appId, metric name, 
> hostname, instanceId. In this situation metric name is normally consist of 
> origin metric name and tag values, like graphite, but unlike Graphite, AMS 
> also doesn't provide series aggregation functions so aggregation should be 
> done from caller side.
> 
> It would be great if Ambari Metrics Collector provides series aggregation 
> functions, like sumSeries / 
> averageSeries / minSeries / maxSeries on Graphite.
> 
> Query outputs: 
> https://gist.github.com/HeartSaVioR/f4f28b5b8b7bf2e5477e59d7fd56090f
> 
> Attached Grafana screenshots to AMBARI-17027. Please refer 
> https://issues.apache.org/jira/browse/AMBARI-17027 for details.
> 
> 
> Diffs
> -
> 
>   ambari-metrics/ambari-metrics-grafana/ambari-metrics/datasource.js 7390aa8 
>   
> ambari-metrics/ambari-metrics-grafana/ambari-metrics/partials/query.editor.html
>  b034c03 
>   ambari-metrics/ambari-metrics-grafana/ambari-metrics/queryCtrl.js 2eb3613 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java
>  1b2d02f 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricStore.java
>  e37bc4d 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricStoreWatcher.java
>  7d49070 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/function/AbstractTimelineMetricsSeriesAggregateFunction.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/function/SeriesAggregateFunction.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/function/TimelineMetricsSeriesAggregateFunction.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/function/TimelineMetricsSeriesAggregateFunctionFactory.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/function/TimelineMetricsSeriesAvgAggregateFunction.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/function/TimelineMetricsSeriesMaxAggregateFunction.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/function/TimelineMetricsSeriesMinAggregateFunction.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/function/TimelineMetricsSeriesSumAggregateFunction.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/webapp/TimelineWebServices.java
>  ee3a097 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TestTimelineMetricStore.java
>  cfd1f58 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricStoreWatcherTest.java
>  a94f4c5 
>   
> 

Re: Review Request 48589: Fix HA enabled logic in the alerts

2016-06-15 Thread Oliver Szabo

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48589/#review137800
---


Ship it!




Ship It!

- Oliver Szabo


On June 14, 2016, 2:15 p.m., Miklos Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48589/
> ---
> 
> (Updated June 14, 2016, 2:15 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Oliver Szabo, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17180
> https://issues.apache.org/jira/browse/AMBARI-17180
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> base_alert.py puts a warning into the log if there are properties referenced 
> in the HA nameservice or the alias which are not present in the 
> configuration. The absence of these properties is an indicator that the HA is 
> not enabled, it is not a cause for warning.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/alerts/base_alert.py 6c8ca5a 
> 
> Diff: https://reviews.apache.org/r/48589/diff/
> 
> 
> Testing
> ---
> 
> Tested on local cluster works fine. Unit tests are running fine too.
> 
> 
> Thanks,
> 
> Miklos Gergely
> 
>



Re: Review Request 48735: Storm service check failed after Ambari upgrade

2016-06-15 Thread Alejandro Fernandez

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48735/#review137808
---



Please include Sriharsha Chintalapani in the code review.

- Alejandro Fernandez


On June 15, 2016, 4:59 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48735/
> ---
> 
> (Updated June 15, 2016, 4:59 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley and Nate Cole.
> 
> 
> Bugs: AMBARI-17258
> https://issues.apache.org/jira/browse/AMBARI-17258
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> ambari-server-2.4.0.0-665.x86_64
> ambari-server --hash
> 
> *Steps*
> # Deploy HDP-2.3.2.0 cluster with Ambari 2.1.2 (secure, non-HA cluster)
> # Upgrade Ambari to 2.4.0.0-665
> # Stop and Start all services
> # Run service check for Storm
> 
> *Result*
> Failure as below:
> {code}
> stderr:   /var/lib/ambari-agent/data/errors-729.txt
> 
> Traceback (most recent call last):
> File 
> "/var/lib/ambari-agent/cache/common-services/STORM/0.9.1/package/scripts/service_check.py",
>  line 79, in 
> ServiceCheck().execute()
> File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 257, in execute
> method(env)
> File 
> "/var/lib/ambari-agent/cache/common-services/STORM/0.9.1/package/scripts/service_check.py",
>  line 70, in service_check
> user=params.storm_user
> File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", 
> line 155, in __init__
> self.env.run()
> File 
> "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
> line 160, in run
> self.run_action(resource, action)
> File 
> "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
> line 124, in run_action
> provider_action()
> File 
> "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py",
>  line 273, in action_run
> tries=self.resource.tries, try_sleep=self.resource.try_sleep)
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
> line 70, in inner
> result = function(command, **kwargs)
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
> line 92, in checked_call
> tries=tries, try_sleep=try_sleep)
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
> line 140, in _call_wrapper
> result = _call(command, **kwargs_copy)
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
> line 293, in _call
> raise Fail(err_msg)
> resource_management.core.exceptions.Fail: Execution of 'storm jar 
> /tmp/wordCount.jar storm.starter.WordCountTopology 
> WordCountid16ac396c_date241516' returned 1. 951  [main] INFO  b.s.u.Utils - 
> Using defaults.yaml from resources
> 1088 [main] INFO  b.s.u.Utils - Using storm.yaml from resources
> 1184 [main] INFO  b.s.u.Utils - Using defaults.yaml from resources
> 1240 [main] INFO  b.s.u.Utils - Using storm.yaml from resources
> 1285 [main] INFO  b.s.StormSubmitter - Generated ZooKeeper secret payload for 
> MD5-digest: -7463484230184273671:-7128399429695564111
> 1287 [main] INFO  b.s.s.a.AuthUtils - Got AutoCreds []
> 1292 [main] WARN  b.s.u.NimbusClient - Using deprecated config nimbus.host 
> for backward compatibility. Please update your storm.yaml so it only has 
> config nimbus.seeds
> 1352 [main] INFO  b.s.u.StormBoundedExponentialBackoffRetry - The 
> baseSleepTimeMs [2000] the maxSleepTimeMs [6] the maxRetries [5]
> 1615 [main] INFO  o.a.s.z.Login - successfully logged in.
> 1650 [main] ERROR o.a.t.t.TSaslTransport - SASL negotiation failure
> javax.security.sasl.SaslException: GSS initiate failed
> at 
> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:212)
>  ~[?:1.7.0_67]
> at 
> org.apache.thrift7.transport.TSaslClientTransport.handleSaslStartMessage(TSaslClientTransport.java:94)
>  ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at org.apache.thrift7.transport.TSaslTransport.open(TSaslTransport.java:271) 
> [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at 
> org.apache.thrift7.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
>  [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at 
> backtype.storm.security.auth.kerberos.KerberosSaslTransportPlugin$1.run(KerberosSaslTransportPlugin.java:145)
>  [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at 
> backtype.storm.security.auth.kerberos.KerberosSaslTransportPlugin$1.run(KerberosSaslTransportPlugin.java:141)
>  [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at java.security.AccessController.doPrivileged(Native Method) ~[?:1.7.0_67]
> at javax.security.auth.Subject.doAs(Subject.java:415) [?:1.7.0_67]
> 

Re: Review Request 48732: (Client) components that are dependencies of services in the stack definitions are always added to blueprint deployments

2016-06-15 Thread Laszlo Puskas

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48732/
---

(Updated June 15, 2016, 7:15 p.m.)


Review request for Ambari, Robert Nettleton, Sandor Magyari, and Sebastian 
Toader.


Bugs: AMBARI-17251
https://issues.apache.org/jira/browse/AMBARI-17251


Repository: ambari


Description
---

Problem:
Component dependencies from stack definitions are automatically added to 
blueprint deployments even the services components belong to are not listed in 
the blueprint.
eg.: SPARK_CLIENT/TEZ_CLIENT components that are listed as dependencies of the 
ATS and RESOURCEMANAGER in the stack definitions are always added to blueprints.

Solution:
Component dependencies defined in stack definitions are only added to 
blueprints if services belonging to the listed components are part of the 
blueprint


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintValidatorImpl.java
 432c6f8 
  
ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintValidatorImplTest.java
 f78d86d 

Diff: https://reviews.apache.org/r/48732/diff/


Testing (updated)
---

Unit tests succeeded
Manually tested on local env.


Thanks,

Laszlo Puskas



Re: Review Request 48741: LogSearch Solr kerberos support

2016-06-15 Thread Oliver Szabo


> On June 15, 2016, 7:11 p.m., Don Bosco Durai wrote:
> > ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/params.py,
> >  line 138
> > 
> >
> > Ideally, we should just use our Hadoop wide HTTP spnego keytab and add 
> > group "hadoop" to logsearch solr user

That is happening here, for every service, spnego keytab/principal is used with 
specific ui configurations


- Oliver


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48741/#review137798
---


On June 15, 2016, 5:37 p.m., Oliver Szabo wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48741/
> ---
> 
> (Updated June 15, 2016, 5:37 p.m.)
> 
> 
> Review request for Ambari, Don Bosco Durai, Miklos Gergely, Robert Levas, 
> Robert Nettleton, Sumit Mohanty, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-16736
> https://issues.apache.org/jira/browse/AMBARI-16736
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Basuc logsearch solr kerberos support.
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/solr_cloud_util.py
>  e0a30ed 
>   ambari-logsearch/ambari-logsearch-logfeeder/pom.xml bbe0bc9 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/OutputSolr.java
>  b14c273 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/resources/logfeeder.properties
>  076c09c 
>   ambari-logsearch/ambari-logsearch-portal/pom.xml e83c75b 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/SolrDaoBase.java
>  cd4ef93 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/resources/logsearch.properties
>  37c8317 
>   ambari-logsearch/ambari-logsearch-solr-client/pom.xml b634928 
>   
> ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/AmbariSolrCloudCLI.java
>  a2da737 
>   
> ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/AmbariSolrCloudClient.java
>  2805b0b 
>   
> ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/AmbariSolrCloudClientBuilder.java
>  0813221 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/alerts.json 
> 85762e0 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logfeeder-env.xml
>  d27067c 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-env.xml
>  3682e5d 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-solr-env.xml
>  2420be0 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/kerberos.json
>  6dd4aa7 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/logfeeder.py
>  ce7f71c 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/logsearch.py
>  2b5fdf7 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/logsearch_common.py
>  d0ac389 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/logsearch_solr.py
>  b55f3d6 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/params.py
>  8a1449d 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logfeeder.py
>  5d1ca85 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch.py
>  368db03 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch_solr.py
>  eac60db 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/status_params.py
>  1efe605 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/templates/logfeeder.properties.j2
>  6a52708 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/templates/logfeeder_jaas.conf.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/templates/logsearch.properties.j2
>  9ada5bf 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/templates/logsearch_jaas.conf.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/templates/logsearch_solr_jaas.conf.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/templates/zoo.cfg.j2
>  1f3808b 
>   
> 

Re: Review Request 48741: LogSearch Solr kerberos support

2016-06-15 Thread Don Bosco Durai


> On June 15, 2016, 7:11 p.m., Don Bosco Durai wrote:
> > ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch_solr.py,
> >  line 82
> > 
> >
> > We should set explicit permission to this path. Only 
> > logsearch-solr-user, logsearch-user, ranger-user and atlas-user should have 
> > write permission. Essentially, in addition to logsearch-solr-user, only 
> > other who need to upload config into zookeeper should have write permission.
> > 
> > I have also seen cases, where if we don't protect zookeeper path, then 
> > if knit'ed user accesses zookeeper, then it will fail.
> 
> Oliver Szabo wrote:
> can it be done in different patch? just because im not sure zkcli of solr 
> is able to set ACLs. if not maybe the users should setup this manually with 
> zookeeper cli anyway.

Yes, we could do it later. We also need to see how hbase is doing it.


- Don Bosco


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48741/#review137798
---


On June 15, 2016, 5:37 p.m., Oliver Szabo wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48741/
> ---
> 
> (Updated June 15, 2016, 5:37 p.m.)
> 
> 
> Review request for Ambari, Don Bosco Durai, Miklos Gergely, Robert Levas, 
> Robert Nettleton, Sumit Mohanty, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-16736
> https://issues.apache.org/jira/browse/AMBARI-16736
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Basuc logsearch solr kerberos support.
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/solr_cloud_util.py
>  e0a30ed 
>   ambari-logsearch/ambari-logsearch-logfeeder/pom.xml bbe0bc9 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/OutputSolr.java
>  b14c273 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/resources/logfeeder.properties
>  076c09c 
>   ambari-logsearch/ambari-logsearch-portal/pom.xml e83c75b 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/SolrDaoBase.java
>  cd4ef93 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/resources/logsearch.properties
>  37c8317 
>   ambari-logsearch/ambari-logsearch-solr-client/pom.xml b634928 
>   
> ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/AmbariSolrCloudCLI.java
>  a2da737 
>   
> ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/AmbariSolrCloudClient.java
>  2805b0b 
>   
> ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/AmbariSolrCloudClientBuilder.java
>  0813221 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/alerts.json 
> 85762e0 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logfeeder-env.xml
>  d27067c 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-env.xml
>  3682e5d 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-solr-env.xml
>  2420be0 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/kerberos.json
>  6dd4aa7 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/logfeeder.py
>  ce7f71c 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/logsearch.py
>  2b5fdf7 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/logsearch_common.py
>  d0ac389 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/logsearch_solr.py
>  b55f3d6 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/params.py
>  8a1449d 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logfeeder.py
>  5d1ca85 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch.py
>  368db03 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch_solr.py
>  eac60db 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/status_params.py
>  1efe605 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/templates/logfeeder.properties.j2
>  6a52708 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/templates/logfeeder_jaas.conf.j2
>  PRE-CREATION 
>   
> 

Review Request 48743: Remove {{atlas_conf_dir}} from HADOOP_CLASSPATH in hive-env.

2016-06-15 Thread Tom Beerbower

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48743/
---

Review request for Ambari and John Speidel.


Bugs: AMBARI-17262
https://issues.apache.org/jira/browse/AMBARI-17262


Repository: ambari


Description
---

In Hive configuration ‘Advanced hive-env’ :


export 
HADOOP_CLASSPATH={{atlas_conf_dir}}:{{atlas_home_dir}}/hook/hive:${HADOOP_CLASSPATH}


remove 


{{atlas_conf_dir}}


Diffs
-

  
ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-env.xml
 0c9aeaf 

Diff: https://reviews.apache.org/r/48743/diff/


Testing
---

Manual test.  Verify configuration.

mvn clean test


Thanks,

Tom Beerbower



Re: Review Request 48735: Storm service check failed after Ambari upgrade

2016-06-15 Thread Sriharsha Chintalapani

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48735/#review137810
---


Ship it!




Ship It!

- Sriharsha Chintalapani


On June 15, 2016, 4:59 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48735/
> ---
> 
> (Updated June 15, 2016, 4:59 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley and Nate Cole.
> 
> 
> Bugs: AMBARI-17258
> https://issues.apache.org/jira/browse/AMBARI-17258
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> ambari-server-2.4.0.0-665.x86_64
> ambari-server --hash
> 
> *Steps*
> # Deploy HDP-2.3.2.0 cluster with Ambari 2.1.2 (secure, non-HA cluster)
> # Upgrade Ambari to 2.4.0.0-665
> # Stop and Start all services
> # Run service check for Storm
> 
> *Result*
> Failure as below:
> {code}
> stderr:   /var/lib/ambari-agent/data/errors-729.txt
> 
> Traceback (most recent call last):
> File 
> "/var/lib/ambari-agent/cache/common-services/STORM/0.9.1/package/scripts/service_check.py",
>  line 79, in 
> ServiceCheck().execute()
> File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 257, in execute
> method(env)
> File 
> "/var/lib/ambari-agent/cache/common-services/STORM/0.9.1/package/scripts/service_check.py",
>  line 70, in service_check
> user=params.storm_user
> File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", 
> line 155, in __init__
> self.env.run()
> File 
> "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
> line 160, in run
> self.run_action(resource, action)
> File 
> "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
> line 124, in run_action
> provider_action()
> File 
> "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py",
>  line 273, in action_run
> tries=self.resource.tries, try_sleep=self.resource.try_sleep)
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
> line 70, in inner
> result = function(command, **kwargs)
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
> line 92, in checked_call
> tries=tries, try_sleep=try_sleep)
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
> line 140, in _call_wrapper
> result = _call(command, **kwargs_copy)
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
> line 293, in _call
> raise Fail(err_msg)
> resource_management.core.exceptions.Fail: Execution of 'storm jar 
> /tmp/wordCount.jar storm.starter.WordCountTopology 
> WordCountid16ac396c_date241516' returned 1. 951  [main] INFO  b.s.u.Utils - 
> Using defaults.yaml from resources
> 1088 [main] INFO  b.s.u.Utils - Using storm.yaml from resources
> 1184 [main] INFO  b.s.u.Utils - Using defaults.yaml from resources
> 1240 [main] INFO  b.s.u.Utils - Using storm.yaml from resources
> 1285 [main] INFO  b.s.StormSubmitter - Generated ZooKeeper secret payload for 
> MD5-digest: -7463484230184273671:-7128399429695564111
> 1287 [main] INFO  b.s.s.a.AuthUtils - Got AutoCreds []
> 1292 [main] WARN  b.s.u.NimbusClient - Using deprecated config nimbus.host 
> for backward compatibility. Please update your storm.yaml so it only has 
> config nimbus.seeds
> 1352 [main] INFO  b.s.u.StormBoundedExponentialBackoffRetry - The 
> baseSleepTimeMs [2000] the maxSleepTimeMs [6] the maxRetries [5]
> 1615 [main] INFO  o.a.s.z.Login - successfully logged in.
> 1650 [main] ERROR o.a.t.t.TSaslTransport - SASL negotiation failure
> javax.security.sasl.SaslException: GSS initiate failed
> at 
> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:212)
>  ~[?:1.7.0_67]
> at 
> org.apache.thrift7.transport.TSaslClientTransport.handleSaslStartMessage(TSaslClientTransport.java:94)
>  ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at org.apache.thrift7.transport.TSaslTransport.open(TSaslTransport.java:271) 
> [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at 
> org.apache.thrift7.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
>  [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at 
> backtype.storm.security.auth.kerberos.KerberosSaslTransportPlugin$1.run(KerberosSaslTransportPlugin.java:145)
>  [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at 
> backtype.storm.security.auth.kerberos.KerberosSaslTransportPlugin$1.run(KerberosSaslTransportPlugin.java:141)
>  [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at java.security.AccessController.doPrivileged(Native Method) ~[?:1.7.0_67]
> at javax.security.auth.Subject.doAs(Subject.java:415) [?:1.7.0_67]
> at 
> 

Re: Review Request 48609: AMBARI-17183: client.properties for Falcon should be configurable via Ambari

2016-06-15 Thread Alejandro Fernandez

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48609/#review137814
---


Ship it!




Ship It!

- Alejandro Fernandez


On June 15, 2016, 5:51 p.m., Venkat Ranganathan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48609/
> ---
> 
> (Updated June 15, 2016, 5:51 p.m.)
> 
> 
> Review request for Ambari.
> 
> 
> Bugs: AMBARI-17183
> https://issues.apache.org/jira/browse/AMBARI-17183
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Currently falcon-client.properties is generated without functionality for 
> user changes.   We should fix it for users of mirroring etc
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon.py
>  b5b3a34 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/params_linux.py
>  22fb691 
>   
> ambari-server/src/main/resources/stacks/HDP/2.1.GlusterFS/services/FALCON/package/scripts/falcon.py
>  9a72af1 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/FALCON/configuration/falcon-client.properties.xml
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/48609/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Venkat Ranganathan
> 
>



Re: Review Request 47941: [AMBARI-16920] Follow up issue for Spark2 stack definition

2016-06-15 Thread Jayush Luniya


> On June 2, 2016, 4:42 p.m., Jayush Luniya wrote:
> > ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/params.py,
> >  line 1
> > 
> >
> > Please add unit tests as mentioned in AMBARI-16864 for SPARK2 service.

There is a patch for UTs


- Jayush


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47941/#review135955
---


On June 13, 2016, 10:45 a.m., Jeff Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47941/
> ---
> 
> (Updated June 13, 2016, 10:45 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-16920
> https://issues.apache.org/jira/browse/AMBARI-16920
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> This a followup ticket for add spark2 stack definition. There's serveral 
> issues:
> 1.  Spark2 thrift server can not started due to miss of 
> spark-thrift-fairscheduler.xml
> 2.  Miss of add spark2 cache file in copy_barball.py
> 3.  Miss the role_commnad_order of spark2
> 4.  conf_select is missign for spark2
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py
>  4eb0015 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py
>  286df8d 
>   
> ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/params.py
>  6925ab5 
>   
> ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/spark_service.py
>  c2385df 
>   ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 
> f6011b0 
> 
> Diff: https://reviews.apache.org/r/47941/diff/
> 
> 
> Testing
> ---
> 
> Manually verified.
> 
> 
> Thanks,
> 
> Jeff Zhang
> 
>



Review Request 48712: AMBARI-17263. Fix for following for Hive Server Interactive : (1). Updates to 'llapstatus' command while querying LLAP app status. (2). Adding validation check for config 'hive.s

2016-06-15 Thread Swapan Shridhar

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48712/
---

Review request for Ambari, Alejandro Fernandez and Sumit Mohanty.


Bugs: AMBARI-17263
https://issues.apache.org/jira/browse/AMBARI-17263


Repository: ambari


Description
---

Fix for following for Hive Server Interactive :
(1). 'llapstatus' command timeout of 0 while querying LLAP app status, update 
retry wait time to 2 seconds and allowing retry on 'COMPLETE' state 
(2). Add validation check for config 'hive.server2.enable.doAs'. the value for 
config shows always stay false.


Diffs
-

  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/alerts/alert_llap_app_status.py
 12c5c19 
  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server_interactive.py
 7de3d49 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
614f0b1 
  
ambari-server/src/test/python/stacks/2.5/common/services-normal-his-2-hosts.json
 4b47cf8 
  ambari-server/src/test/python/stacks/2.5/common/test_stack_advisor.py 4085ad6 

Diff: https://reviews.apache.org/r/48712/diff/


Testing
---

Added Validation UT.
- Python UT passes.


Thanks,

Swapan Shridhar



Re: Review Request 48741: LogSearch Solr kerberos support

2016-06-15 Thread Oliver Szabo


> On June 15, 2016, 7:11 p.m., Don Bosco Durai wrote:
> > ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch_solr.py,
> >  line 82
> > 
> >
> > We should set explicit permission to this path. Only 
> > logsearch-solr-user, logsearch-user, ranger-user and atlas-user should have 
> > write permission. Essentially, in addition to logsearch-solr-user, only 
> > other who need to upload config into zookeeper should have write permission.
> > 
> > I have also seen cases, where if we don't protect zookeeper path, then 
> > if knit'ed user accesses zookeeper, then it will fail.

can it be done in different patch? just because im not sure zkcli of solr is 
able to set ACLs. if not maybe the users should setup this manually with 
zookeeper cli anyway.


- Oliver


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48741/#review137798
---


On June 15, 2016, 5:37 p.m., Oliver Szabo wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48741/
> ---
> 
> (Updated June 15, 2016, 5:37 p.m.)
> 
> 
> Review request for Ambari, Don Bosco Durai, Miklos Gergely, Robert Levas, 
> Robert Nettleton, Sumit Mohanty, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-16736
> https://issues.apache.org/jira/browse/AMBARI-16736
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Basuc logsearch solr kerberos support.
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/solr_cloud_util.py
>  e0a30ed 
>   ambari-logsearch/ambari-logsearch-logfeeder/pom.xml bbe0bc9 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/OutputSolr.java
>  b14c273 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/resources/logfeeder.properties
>  076c09c 
>   ambari-logsearch/ambari-logsearch-portal/pom.xml e83c75b 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/SolrDaoBase.java
>  cd4ef93 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/resources/logsearch.properties
>  37c8317 
>   ambari-logsearch/ambari-logsearch-solr-client/pom.xml b634928 
>   
> ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/AmbariSolrCloudCLI.java
>  a2da737 
>   
> ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/AmbariSolrCloudClient.java
>  2805b0b 
>   
> ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/AmbariSolrCloudClientBuilder.java
>  0813221 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/alerts.json 
> 85762e0 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logfeeder-env.xml
>  d27067c 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-env.xml
>  3682e5d 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-solr-env.xml
>  2420be0 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/kerberos.json
>  6dd4aa7 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/logfeeder.py
>  ce7f71c 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/logsearch.py
>  2b5fdf7 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/logsearch_common.py
>  d0ac389 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/logsearch_solr.py
>  b55f3d6 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/params.py
>  8a1449d 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logfeeder.py
>  5d1ca85 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch.py
>  368db03 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch_solr.py
>  eac60db 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/status_params.py
>  1efe605 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/templates/logfeeder.properties.j2
>  6a52708 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/templates/logfeeder_jaas.conf.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/templates/logsearch.properties.j2
>  9ada5bf 
>   
> 

Re: Review Request 48722: Reduce the idle time before first command from next stage is executed on a host

2016-06-15 Thread Alejandro Fernandez

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48722/#review137827
---


Ship it!




Ship It!

- Alejandro Fernandez


On June 15, 2016, 2:55 p.m., Sebastian Toader wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48722/
> ---
> 
> (Updated June 15, 2016, 2:55 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Laszlo Puskas, Robert Levas, 
> Sandor Magyari, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17248
> https://issues.apache.org/jira/browse/AMBARI-17248
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Commands to be executed by ambari-agents are being sent down by the server in 
> the response message to agent heartbeat messages. 
> The server processes the received heartbeat, it checks if there are next 
> commands scheduled to be executed by ambari-agent and adds those to the 
> heartbeat response for the ambari-agent.
> The server organises the commands that can be executed in parallel into 
> stages. Ambari server ensures that only the commands of a single stage is 
> scheduled to be executed by the agent and starts scheduling the commands of 
> the next stage only after all commands of current stage has finished 
> successfully.
> The processing of command status received with the heartbeat message happens 
> asynchronously to heartbeat response in HeartBeatProcessor and 
> ActionScheduler creation thus when the heartbeat response is created the 
> commands for the next stage are not scheduled yet. This means that the next 
> commands will be sent to agent only with the next heartbeat.
> Agents currently sends a heartbeat to the server on command a completion or 
> at a timeout = self.netutil.HEARTBEAT_IDDLE_INTERVAL_SEC – 
> self.netutil.MINIMUM_INTERVAL_BETWEEN_HEARTBEATS interval which is ~10 
> seconds if there are no commands to be executed.
> This means that when the server receives a heartbeat triggered by the 
> completion of the last command from the current stage the server will send 
> the commands for the next stage only 10 seconds later when the next heartbeat 
> is received. This leads to agents spending considerable amount of time idle 
> when there are multiple stages to be executed.
> Agents should heartbeat at a higher rate while there are still pending stages 
> to be executed.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/Controller.py e981a76 
>   ambari-agent/src/main/python/ambari_agent/NetUtil.py 80bf3ae 
>   ambari-agent/src/test/python/ambari_agent/TestNetUtil.py d72e319 
>   ambari-agent/src/test/python/ambari_agent/examples/ControllerTester.py 
> 8103872 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java
>  35a37e3 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatResponse.java
>  1ab7ae9 
>   ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java 
> ac0ddd2 
>   ambari-server/src/main/java/org/apache/ambari/server/state/Clusters.java 
> bd9de13 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
>  3d2388e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClustersImpl.java
>  c26e1e9 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClusterImplTest.java
>  627ade9 
> 
> Diff: https://reviews.apache.org/r/48722/diff/
> 
> 
> Testing
> ---
> 
> Manual testing.
> 
> Unit tests in succeeded.
> 
> 
> Thanks,
> 
> Sebastian Toader
> 
>



Re: Review Request 48750: AMBARI-17150 : Ambari Metrics components packages do not have vendor before upgrade, but for new versions it appears

2016-06-15 Thread Ajit Kumar

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48750/#review137834
---


Ship it!




Please make sure while building metrics, no other version of surefire is being 
linked.


ambari-metrics/ambari-metrics-timelineservice/pom.xml (line 711)


As far as I remember, I had changed pom.version to project.version because 
of a build warning. Can you please check if you are not getting any warning 
because of this revert?


- Ajit Kumar


On June 15, 2016, 8:29 p.m., Aravindan Vijayan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48750/
> ---
> 
> (Updated June 15, 2016, 8:29 p.m.)
> 
> 
> Review request for Ambari, Ajit Kumar, Dmytro Sen, Sumit Mohanty, and Sid 
> Wagle.
> 
> 
> Bugs: AMBARI-17150
> https://issues.apache.org/jira/browse/AMBARI-17150
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In December 2015 changes were made such that Ambari metrics project's parent 
> is Ambari. This change was done to make sure the surefire plugin library is 
> inherited. 
> 
> However, AMS pom.xml did not have any Vendor (organization) information, and 
> hence it inherited from Ambari project ('Apache Software Foundation'). Hence, 
> the released AMS packages have no Vendor and the new ones will have it thus 
> causing upgrade command to fail 
> 
> suse1101:~ # zypper up ambari-metrics-hadoop-sink
> Loading repository data...
> Reading installed packages...
> There is an update candidate for 'ambari-metrics-hadoop-sink', but it is from 
> different vendor. Use 'zypper install 
> ambari-metrics-hadoop-sink-2.4.0.0-569.x86_64' to install this candidate.
> Resolving package dependencies...
> Nothing to do.
> 
> Fix
> Remove the hierarchy and add surefire plugin dependency in Ambari metrics 
> project.
> 
> Related review board
> https://reviews.apache.org/r/41269
> 
> 
> Diffs
> -
> 
>   ambari-metrics/ambari-metrics-common/pom.xml af37d28 
>   ambari-metrics/ambari-metrics-timelineservice/pom.xml ed612cc 
>   ambari-metrics/pom.xml 36d817e 
> 
> Diff: https://reviews.apache.org/r/48750/diff/
> 
> 
> Testing
> ---
> 
> mvn clean package
> 
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Ambari Main ... SUCCESS [9.364s]
> [INFO] Apache Ambari Project POM . SUCCESS [0.028s]
> [INFO] Ambari Web  SUCCESS [24.758s]
> [INFO] Ambari Views .. SUCCESS [1.338s]
> [INFO] Ambari Admin View . SUCCESS [9.822s]
> [INFO] ambari-metrics  SUCCESS [0.586s]
> [INFO] Ambari Metrics Common . SUCCESS [1.081s]
> [INFO] Ambari Metrics Hadoop Sink  SUCCESS [0.955s]
> [INFO] Ambari Metrics Flume Sink . SUCCESS [0.453s]
> [INFO] Ambari Metrics Kafka Sink . SUCCESS [0.562s]
> [INFO] Ambari Metrics Storm Sink . SUCCESS [1.389s]
> [INFO] Ambari Metrics Collector .. SUCCESS [2:17.967s]
> [INFO] Ambari Metrics Monitor  SUCCESS [2.207s]
> [INFO] Ambari Metrics Grafana  SUCCESS [54.849s]
> [INFO] Ambari Metrics Assembly ... SUCCESS [2:22.821s]
> [INFO] Ambari Server . SUCCESS [1:31.407s]
> [INFO] Ambari Functional Tests ... SUCCESS [1.977s]
> [INFO] Ambari Agent .. SUCCESS [23.784s]
> [INFO] Ambari Client . SUCCESS [0.032s]
> [INFO] Ambari Python Client .. SUCCESS [0.618s]
> [INFO] Ambari Groovy Client .. SUCCESS [1.972s]
> [INFO] Ambari Shell .. SUCCESS [0.044s]
> [INFO] Ambari Python Shell ... SUCCESS [0.463s]
> [INFO] Ambari Groovy Shell ... SUCCESS [0.830s]
> [INFO] ambari-logsearch .. SUCCESS [0.233s]
> [INFO] Ambari Logsearch Appender . SUCCESS [0.316s]
> [INFO] Ambari Logsearch Solr Client .. SUCCESS [1.024s]
> [INFO] Ambari Logsearch Portal ... SUCCESS [5.953s]
> [INFO] Ambari Logsearch Log Feeder ... SUCCESS [1.429s]
> [INFO] Ambari Logsearch Assembly . SUCCESS [0.047s]
> [INFO] 
> 

Re: Review Request 48750: AMBARI-17150 : Ambari Metrics components packages do not have vendor before upgrade, but for new versions it appears

2016-06-15 Thread Aravindan Vijayan

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48750/
---

(Updated June 15, 2016, 8:59 p.m.)


Review request for Ambari, Ajit Kumar, Dmytro Sen, Sumit Mohanty, and Sid Wagle.


Bugs: AMBARI-17150
https://issues.apache.org/jira/browse/AMBARI-17150


Repository: ambari


Description
---

In December 2015 changes were made such that Ambari metrics project's parent is 
Ambari. This change was done to make sure the surefire plugin library is 
inherited. 

However, AMS pom.xml did not have any Vendor (organization) information, and 
hence it inherited from Ambari project ('Apache Software Foundation'). Hence, 
the released AMS packages have no Vendor and the new ones will have it thus 
causing upgrade command to fail 

suse1101:~ # zypper up ambari-metrics-hadoop-sink
Loading repository data...
Reading installed packages...
There is an update candidate for 'ambari-metrics-hadoop-sink', but it is from 
different vendor. Use 'zypper install 
ambari-metrics-hadoop-sink-2.4.0.0-569.x86_64' to install this candidate.
Resolving package dependencies...
Nothing to do.

Fix
Remove the hierarchy and add surefire plugin dependency in Ambari metrics 
project.

Related review board
https://reviews.apache.org/r/41269


Diffs (updated)
-

  ambari-metrics/ambari-metrics-common/pom.xml af37d28 
  ambari-metrics/pom.xml 36d817e 

Diff: https://reviews.apache.org/r/48750/diff/


Testing
---

mvn clean package

[INFO] 
[INFO] Reactor Summary:
[INFO]
[INFO] Ambari Main ... SUCCESS [9.364s]
[INFO] Apache Ambari Project POM . SUCCESS [0.028s]
[INFO] Ambari Web  SUCCESS [24.758s]
[INFO] Ambari Views .. SUCCESS [1.338s]
[INFO] Ambari Admin View . SUCCESS [9.822s]
[INFO] ambari-metrics  SUCCESS [0.586s]
[INFO] Ambari Metrics Common . SUCCESS [1.081s]
[INFO] Ambari Metrics Hadoop Sink  SUCCESS [0.955s]
[INFO] Ambari Metrics Flume Sink . SUCCESS [0.453s]
[INFO] Ambari Metrics Kafka Sink . SUCCESS [0.562s]
[INFO] Ambari Metrics Storm Sink . SUCCESS [1.389s]
[INFO] Ambari Metrics Collector .. SUCCESS [2:17.967s]
[INFO] Ambari Metrics Monitor  SUCCESS [2.207s]
[INFO] Ambari Metrics Grafana  SUCCESS [54.849s]
[INFO] Ambari Metrics Assembly ... SUCCESS [2:22.821s]
[INFO] Ambari Server . SUCCESS [1:31.407s]
[INFO] Ambari Functional Tests ... SUCCESS [1.977s]
[INFO] Ambari Agent .. SUCCESS [23.784s]
[INFO] Ambari Client . SUCCESS [0.032s]
[INFO] Ambari Python Client .. SUCCESS [0.618s]
[INFO] Ambari Groovy Client .. SUCCESS [1.972s]
[INFO] Ambari Shell .. SUCCESS [0.044s]
[INFO] Ambari Python Shell ... SUCCESS [0.463s]
[INFO] Ambari Groovy Shell ... SUCCESS [0.830s]
[INFO] ambari-logsearch .. SUCCESS [0.233s]
[INFO] Ambari Logsearch Appender . SUCCESS [0.316s]
[INFO] Ambari Logsearch Solr Client .. SUCCESS [1.024s]
[INFO] Ambari Logsearch Portal ... SUCCESS [5.953s]
[INFO] Ambari Logsearch Log Feeder ... SUCCESS [1.429s]
[INFO] Ambari Logsearch Assembly . SUCCESS [0.047s]
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 8:38.840s
[INFO] Finished at: Wed Jun 15 13:13:20 PDT 2016
[INFO] Final Memory: 206M/1529M
[INFO] 


Thanks,

Aravindan Vijayan



Re: Review Request 48395: AMBARI-17027: Metrics Collector API: Introduce basic series aggregation functions

2016-06-15 Thread Jungtaek Lim


> On 6 15, 2016, 6:16 오후, Prajwal Rao wrote:
> > ambari-metrics/ambari-metrics-grafana/ambari-metrics/queryCtrl.js, line 40
> > 
> >
> > There seems to be no way to disable Series Aggregation. We need to have 
> > a "none" option, and "sum" shouldn't be set as the default option.

There's check button on the UI (target.shouldAggregateSeries) and series 
aggregation only works when button is checked. Please let me know there's a bug 
on checking enable/disable series aggregation. Thanks!


- Jungtaek


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48395/#review137790
---


On 6 10, 2016, 12:50 오전, Jungtaek Lim wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48395/
> ---
> 
> (Updated 6 10, 2016, 12:50 오전)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Dmytro Sen, Prajwal Rao, 
> Sriharsha Chintalapani, Sid Wagle, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-17027
> https://issues.apache.org/jira/browse/AMBARI-17027
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> AMS doesn't provide tag so metric is identified by appId, metric name, 
> hostname, instanceId. In this situation metric name is normally consist of 
> origin metric name and tag values, like graphite, but unlike Graphite, AMS 
> also doesn't provide series aggregation functions so aggregation should be 
> done from caller side.
> 
> It would be great if Ambari Metrics Collector provides series aggregation 
> functions, like sumSeries / 
> averageSeries / minSeries / maxSeries on Graphite.
> 
> Query outputs: 
> https://gist.github.com/HeartSaVioR/f4f28b5b8b7bf2e5477e59d7fd56090f
> 
> Attached Grafana screenshots to AMBARI-17027. Please refer 
> https://issues.apache.org/jira/browse/AMBARI-17027 for details.
> 
> 
> Diffs
> -
> 
>   ambari-metrics/ambari-metrics-grafana/ambari-metrics/datasource.js 7390aa8 
>   
> ambari-metrics/ambari-metrics-grafana/ambari-metrics/partials/query.editor.html
>  b034c03 
>   ambari-metrics/ambari-metrics-grafana/ambari-metrics/queryCtrl.js 2eb3613 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java
>  1b2d02f 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricStore.java
>  e37bc4d 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricStoreWatcher.java
>  7d49070 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/function/AbstractTimelineMetricsSeriesAggregateFunction.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/function/SeriesAggregateFunction.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/function/TimelineMetricsSeriesAggregateFunction.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/function/TimelineMetricsSeriesAggregateFunctionFactory.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/function/TimelineMetricsSeriesAvgAggregateFunction.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/function/TimelineMetricsSeriesMaxAggregateFunction.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/function/TimelineMetricsSeriesMinAggregateFunction.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/function/TimelineMetricsSeriesSumAggregateFunction.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/webapp/TimelineWebServices.java
>  ee3a097 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TestTimelineMetricStore.java
>  cfd1f58 
>   
> 

Re: Review Request 48739: Remove CascadeType in ServiceComponentHistoryEntity for ServiceComponentDesiredStateEntity

2016-06-15 Thread Nahappan Somasundaram

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48739/#review137903
---


Ship it!




Ship It!

- Nahappan Somasundaram


On June 15, 2016, 2:19 p.m., Ajit Kumar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48739/
> ---
> 
> (Updated June 15, 2016, 2:19 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, 
> and Sumit Gupta.
> 
> 
> Bugs: AMBARI-17260
> https://issues.apache.org/jira/browse/AMBARI-17260
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Remove CascadeType in ServiceComponentHistoryEntity for 
> ServiceComponentDesiredStateEntity
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/ServiceComponentHistoryEntity.java
>  e7fef71 
> 
> Diff: https://reviews.apache.org/r/48739/diff/
> 
> 
> Testing
> ---
> 
> Tested manually by deleting service
> 
> 
> Thanks,
> 
> Ajit Kumar
> 
>



Re: Review Request 48727: Log Search default log levels can not be altered

2016-06-15 Thread Miklos Gergely


> On June 15, 2016, 7:38 p.m., Don Bosco Durai wrote:
> > ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/UserConfigSolrDao.java,
> >  line 133
> > 
> >
> > Not sure I understood this use case. The defaults from the properties 
> > is only used for bootstraping values in the Solr. Once it is in the Solr, 
> > users might change the default values via LogSearch UI. If we overwrite the 
> > values in Solr on each startup, then any user changes will be lost.
> > 
> > I assume, this requirement is coming due to support for new services 
> > that could be added later. If so, we might have to change the logic for 
> > bootstraping the initial values.
> > 
> > If this is the case, we might have to discuss more.

logsearch.logfeeder.include.default.level can be modified on ambari after Log 
Search was installed. In this case the user would expect to see the effect of 
this modification. As I see there are 3 ways to solve this:

1. Allow the users to modify the config only by altering 
logsearch.logfeeder.include.default.level
2. Allow the users to modify the config only on the LogSearch UI, and remove 
logsearch.logfeeder.include.default.level from the property list, and the 
bootstrap should be done with some constant
3. If there is a way to make logsearch.logfeeder.include.default.level 
unmodifiable after the installation is done, then this would fix the issue too


- Miklos


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48727/#review137811
---


On June 15, 2016, 12:46 p.m., Miklos Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48727/
> ---
> 
> (Updated June 15, 2016, 12:46 p.m.)
> 
> 
> Review request for Ambari, Don Bosco Durai, Dharmesh Makwana, Oliver Szabo, 
> Robert Nettleton, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17254
> https://issues.apache.org/jira/browse/AMBARI-17254
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> The default log levels (log levels to include) can not be altered, forever 
> the ones set during the installation (more precisely: the one which is the 
> value of the property during the first run of the portal) would be in effect.
> 
> Currently the portal checks if there is a configuration, and does nothing if 
> there is. After this change it will remove any existing configuration upon 
> starting, and replace it with a newly generated configuration.
> 
> 
> Diffs
> -
> 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/SolrDaoBase.java
>  147e148 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/UserConfigSolrDao.java
>  edf1dcc 
> 
> Diff: https://reviews.apache.org/r/48727/diff/
> 
> 
> Testing
> ---
> 
> Tested on local cluster.
> 
> 
> Thanks,
> 
> Miklos Gergely
> 
>



Re: Review Request 48670: Return well formatted error response while deleting host with clients installed.

2016-06-15 Thread Ajit Kumar


> On June 15, 2016, 9:23 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostResourceProvider.java,
> >  line 930
> > 
> >
> > I also talked to Sumit, we're not going to distinguish between clients 
> > so any components should cause a failure.

Thanks. As we won't be differentiating b/w client and other components, this 
patch will result in error message for clients as for other components. I'll 
push this patch.


- Ajit


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48670/#review137853
---


On June 14, 2016, 12:29 a.m., Ajit Kumar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48670/
> ---
> 
> (Updated June 14, 2016, 12:29 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jayush Luniya, and Sumit 
> Mohanty.
> 
> 
> Bugs: AMBARI-17210
> https://issues.apache.org/jira/browse/AMBARI-17210
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Return well formatted error response while deleting host with clients 
> installed.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostResourceProvider.java
>  46fc65295f72bae39c534cd9684f2d76d1e30224 
> 
> Diff: https://reviews.apache.org/r/48670/diff/
> 
> 
> Testing
> ---
> 
> Manual testing
> 
> delete 
> http://c6401.ambari.apache.org:8080/api/v1/clusters/c1/hosts/c6403.ambari.apache.org
> {
>   "status" : 500,
>   "message" : "org.apache.ambari.server.controller.spi.SystemException: An 
> internal system exception occurred: Cannot remove host 
> c6403.ambari.apache.org from c1.  The following roles exist, and these 
> components must be stopped if running, and then deleted: HDFS_CLIENT, 
> MAPREDUCE2_CLIENT, YARN_CLIENT, ZOOKEEPER_CLIENT"
> }
> 
> 
> Thanks,
> 
> Ajit Kumar
> 
>



Re: Review Request 48727: Log Search default log levels can not be altered

2016-06-15 Thread Don Bosco Durai


> On June 15, 2016, 7:38 p.m., Don Bosco Durai wrote:
> > ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/UserConfigSolrDao.java,
> >  line 133
> > 
> >
> > Not sure I understood this use case. The defaults from the properties 
> > is only used for bootstraping values in the Solr. Once it is in the Solr, 
> > users might change the default values via LogSearch UI. If we overwrite the 
> > values in Solr on each startup, then any user changes will be lost.
> > 
> > I assume, this requirement is coming due to support for new services 
> > that could be added later. If so, we might have to change the logic for 
> > bootstraping the initial values.
> > 
> > If this is the case, we might have to discuss more.
> 
> Miklos Gergely wrote:
> logsearch.logfeeder.include.default.level can be modified on ambari after 
> Log Search was installed. In this case the user would expect to see the 
> effect of this modification. As I see there are 3 ways to solve this:
> 
> 1. Allow the users to modify the config only by altering 
> logsearch.logfeeder.include.default.level
> 2. Allow the users to modify the config only on the LogSearch UI, and 
> remove logsearch.logfeeder.include.default.level from the property list, and 
> the bootstrap should be done with some constant
> 3. If there is a way to make logsearch.logfeeder.include.default.level 
> unmodifiable after the installation is done, then this would fix the issue too

I would prefer option #3, because I feel in some environment they want all 
logs. Is it possible to do it?


- Don Bosco


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48727/#review137811
---


On June 15, 2016, 12:46 p.m., Miklos Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48727/
> ---
> 
> (Updated June 15, 2016, 12:46 p.m.)
> 
> 
> Review request for Ambari, Don Bosco Durai, Dharmesh Makwana, Oliver Szabo, 
> Robert Nettleton, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17254
> https://issues.apache.org/jira/browse/AMBARI-17254
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> The default log levels (log levels to include) can not be altered, forever 
> the ones set during the installation (more precisely: the one which is the 
> value of the property during the first run of the portal) would be in effect.
> 
> Currently the portal checks if there is a configuration, and does nothing if 
> there is. After this change it will remove any existing configuration upon 
> starting, and replace it with a newly generated configuration.
> 
> 
> Diffs
> -
> 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/SolrDaoBase.java
>  147e148 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/UserConfigSolrDao.java
>  edf1dcc 
> 
> Diff: https://reviews.apache.org/r/48727/diff/
> 
> 
> Testing
> ---
> 
> Tested on local cluster.
> 
> 
> Thanks,
> 
> Miklos Gergely
> 
>



Re: Review Request 48741: LogSearch Solr kerberos support

2016-06-15 Thread Don Bosco Durai


> On June 15, 2016, 7:11 p.m., Don Bosco Durai wrote:
> > ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch_solr.py,
> >  line 82
> > 
> >
> > We should set explicit permission to this path. Only 
> > logsearch-solr-user, logsearch-user, ranger-user and atlas-user should have 
> > write permission. Essentially, in addition to logsearch-solr-user, only 
> > other who need to upload config into zookeeper should have write permission.
> > 
> > I have also seen cases, where if we don't protect zookeeper path, then 
> > if knit'ed user accesses zookeeper, then it will fail.
> 
> Oliver Szabo wrote:
> can it be done in different patch? just because im not sure zkcli of solr 
> is able to set ACLs. if not maybe the users should setup this manually with 
> zookeeper cli anyway.
> 
> Don Bosco Durai wrote:
> Yes, we could do it later. We also need to see how hbase is doing it.
> 
> Oliver Szabo wrote:
> If zkcli cannot do that we can still use the logsearch-solr-client to set 
> ACLs programmatically

You are correct. logsearch-solr-client has the zookeeper client. We just have 
to run it with the appropriate user permisison to change the permission of the 
zookeeper path


- Don Bosco


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48741/#review137798
---


On June 15, 2016, 5:37 p.m., Oliver Szabo wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48741/
> ---
> 
> (Updated June 15, 2016, 5:37 p.m.)
> 
> 
> Review request for Ambari, Don Bosco Durai, Miklos Gergely, Robert Levas, 
> Robert Nettleton, Sumit Mohanty, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-16736
> https://issues.apache.org/jira/browse/AMBARI-16736
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Basuc logsearch solr kerberos support.
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/solr_cloud_util.py
>  e0a30ed 
>   ambari-logsearch/ambari-logsearch-logfeeder/pom.xml bbe0bc9 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/OutputSolr.java
>  b14c273 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/resources/logfeeder.properties
>  076c09c 
>   ambari-logsearch/ambari-logsearch-portal/pom.xml e83c75b 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/SolrDaoBase.java
>  cd4ef93 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/resources/logsearch.properties
>  37c8317 
>   ambari-logsearch/ambari-logsearch-solr-client/pom.xml b634928 
>   
> ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/AmbariSolrCloudCLI.java
>  a2da737 
>   
> ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/AmbariSolrCloudClient.java
>  2805b0b 
>   
> ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/AmbariSolrCloudClientBuilder.java
>  0813221 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/alerts.json 
> 85762e0 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logfeeder-env.xml
>  d27067c 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-env.xml
>  3682e5d 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-solr-env.xml
>  2420be0 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/kerberos.json
>  6dd4aa7 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/logfeeder.py
>  ce7f71c 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/logsearch.py
>  2b5fdf7 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/logsearch_common.py
>  d0ac389 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/logsearch_solr.py
>  b55f3d6 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/params.py
>  8a1449d 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logfeeder.py
>  5d1ca85 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch.py
>  368db03 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch_solr.py
>  eac60db 
>   
> 

Re: Review Request 48670: Return well formatted error response while deleting host with clients installed.

2016-06-15 Thread Alejandro Fernandez

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48670/#review137860
---


Ship it!




Ship It!

- Alejandro Fernandez


On June 14, 2016, 12:29 a.m., Ajit Kumar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48670/
> ---
> 
> (Updated June 14, 2016, 12:29 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jayush Luniya, and Sumit 
> Mohanty.
> 
> 
> Bugs: AMBARI-17210
> https://issues.apache.org/jira/browse/AMBARI-17210
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Return well formatted error response while deleting host with clients 
> installed.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostResourceProvider.java
>  46fc65295f72bae39c534cd9684f2d76d1e30224 
> 
> Diff: https://reviews.apache.org/r/48670/diff/
> 
> 
> Testing
> ---
> 
> Manual testing
> 
> delete 
> http://c6401.ambari.apache.org:8080/api/v1/clusters/c1/hosts/c6403.ambari.apache.org
> {
>   "status" : 500,
>   "message" : "org.apache.ambari.server.controller.spi.SystemException: An 
> internal system exception occurred: Cannot remove host 
> c6403.ambari.apache.org from c1.  The following roles exist, and these 
> components must be stopped if running, and then deleted: HDFS_CLIENT, 
> MAPREDUCE2_CLIENT, YARN_CLIENT, ZOOKEEPER_CLIENT"
> }
> 
> 
> Thanks,
> 
> Ajit Kumar
> 
>



Re: Review Request 48712: AMBARI-17263. Fix for following for Hive Server Interactive : (1). Updates to 'llapstatus' command while querying LLAP app status. (2). Adding validation check for config 'hi

2016-06-15 Thread Sumit Mohanty

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48712/#review137829
---


Ship it!




Ship It!

- Sumit Mohanty


On June 15, 2016, 6:47 p.m., Swapan Shridhar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48712/
> ---
> 
> (Updated June 15, 2016, 6:47 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17263
> https://issues.apache.org/jira/browse/AMBARI-17263
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Fix for following for Hive Server Interactive :
> (1). 'llapstatus' command timeout of 0 while querying LLAP app status, update 
> retry wait time to 2 seconds and allowing retry on 'COMPLETE' state 
> (2). Add validation check for config 'hive.server2.enable.doAs'. the value 
> for config shows always stay false.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/alerts/alert_llap_app_status.py
>  12c5c19 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server_interactive.py
>  7de3d49 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
> 614f0b1 
>   
> ambari-server/src/test/python/stacks/2.5/common/services-normal-his-2-hosts.json
>  4b47cf8 
>   ambari-server/src/test/python/stacks/2.5/common/test_stack_advisor.py 
> 4085ad6 
> 
> Diff: https://reviews.apache.org/r/48712/diff/
> 
> 
> Testing
> ---
> 
> Added Validation UT.
> - Python UT passes.
> 
> 
> Thanks,
> 
> Swapan Shridhar
> 
>



Re: Review Request 47656: AMBARI-12885 - Dynamic stack extensions - install and upgrade support for custom services

2016-06-15 Thread Alejandro Fernandez

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47656/#review137828
---


Ship it!




Tim, let me know if you want me to commit this to trunk.

- Alejandro Fernandez


On June 15, 2016, 2:08 p.m., Tim Thorpe wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47656/
> ---
> 
> (Updated June 15, 2016, 2:08 p.m.)
> 
> 
> Review request for Ambari, Alexander Denissov, Alejandro Fernandez, bhuvnesh 
> chaudhary, Jayush Luniya, Oleksandr Diachenko, Sumit Mohanty, Srimanth 
> Gunturi, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-12885
> https://issues.apache.org/jira/browse/AMBARI-12885
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> The purpose of this proposal is to facilitate adding custom services to an 
> existing stack. Ideally this would support adding and upgrading custom 
> services separately from the core services defined in the stack. In 
> particular we are looking at custom services that need to support several 
> different stacks (different distributions of Ambari). The release cycle of 
> the custom services may be different from that of the core stack; that is, a 
> custom service may be upgraded at a different rate than the core distribution 
> itself and may be upgraded multiple times within the lifespan of a single 
> release of the core distribution.
> 
> One possible approach to handling this would be dynamically extending a stack 
> (after install time). It would be best to extend the stack in packages where 
> a stack extension package can have one or more custom services.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py 
> fc1b72a 
>   ambari-agent/src/main/python/ambari_agent/FileCache.py b7c5dee 
>   ambari-server/conf/unix/ambari.properties a88a025 
>   ambari-server/src/main/assemblies/server.xml 1560d8d 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/ExtensionLinkResourceDefinition.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/ExtensionResourceDefinition.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/ExtensionVersionResourceDefinition.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/ResourceInstanceFactoryImpl.java
>  cf7c391 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java
>  f0928cf 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/ExtensionLinksService.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/ExtensionsService.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/StacksService.java
>  557ce98 
>   
> ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
>  2b7fca0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementController.java
>  b488af3 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  ba93d25 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/ExtensionLinkRequest.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/ExtensionLinkResponse.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/ExtensionRequest.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/ExtensionResponse.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/ExtensionVersionRequest.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/ExtensionVersionResponse.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractControllerResourceProvider.java
>  3721113 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/Extension.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ExtensionLinkResourceProvider.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ExtensionResourceProvider.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ExtensionVersionResourceProvider.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/spi/Resource.java
>  99e4ccd 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/ExtensionDAO.java
>  PRE-CREATION 
>   
> 

Review Request 48750: AMBARI-17150 : Ambari Metrics components packages do not have vendor before upgrade, but for new versions it appears

2016-06-15 Thread Aravindan Vijayan

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48750/
---

Review request for Ambari, Ajit Kumar, Dmytro Sen, Sumit Mohanty, and Sid Wagle.


Bugs: AMBARI-17150
https://issues.apache.org/jira/browse/AMBARI-17150


Repository: ambari


Description
---

In December 2015 changes were made such that Ambari metrics project's parent is 
Ambari. This change was done to make sure the surefire plugin library is 
inherited. 

However, AMS pom.xml did not have any Vendor (organization) information, and 
hence it inherited from Ambari project ('Apache Software Foundation'). Hence, 
the released AMS packages have no Vendor and the new ones will have it thus 
causing upgrade command to fail 

suse1101:~ # zypper up ambari-metrics-hadoop-sink
Loading repository data...
Reading installed packages...
There is an update candidate for 'ambari-metrics-hadoop-sink', but it is from 
different vendor. Use 'zypper install 
ambari-metrics-hadoop-sink-2.4.0.0-569.x86_64' to install this candidate.
Resolving package dependencies...
Nothing to do.

Fix
Remove the hierarchy and add surefire plugin dependency in Ambari metrics 
project.

Related review board
https://reviews.apache.org/r/41269


Diffs
-

  ambari-metrics/ambari-metrics-common/pom.xml af37d28 
  ambari-metrics/ambari-metrics-timelineservice/pom.xml ed612cc 
  ambari-metrics/pom.xml 36d817e 

Diff: https://reviews.apache.org/r/48750/diff/


Testing
---

mvn clean package

[INFO] 
[INFO] Reactor Summary:
[INFO]
[INFO] Ambari Main ... SUCCESS [9.364s]
[INFO] Apache Ambari Project POM . SUCCESS [0.028s]
[INFO] Ambari Web  SUCCESS [24.758s]
[INFO] Ambari Views .. SUCCESS [1.338s]
[INFO] Ambari Admin View . SUCCESS [9.822s]
[INFO] ambari-metrics  SUCCESS [0.586s]
[INFO] Ambari Metrics Common . SUCCESS [1.081s]
[INFO] Ambari Metrics Hadoop Sink  SUCCESS [0.955s]
[INFO] Ambari Metrics Flume Sink . SUCCESS [0.453s]
[INFO] Ambari Metrics Kafka Sink . SUCCESS [0.562s]
[INFO] Ambari Metrics Storm Sink . SUCCESS [1.389s]
[INFO] Ambari Metrics Collector .. SUCCESS [2:17.967s]
[INFO] Ambari Metrics Monitor  SUCCESS [2.207s]
[INFO] Ambari Metrics Grafana  SUCCESS [54.849s]
[INFO] Ambari Metrics Assembly ... SUCCESS [2:22.821s]
[INFO] Ambari Server . SUCCESS [1:31.407s]
[INFO] Ambari Functional Tests ... SUCCESS [1.977s]
[INFO] Ambari Agent .. SUCCESS [23.784s]
[INFO] Ambari Client . SUCCESS [0.032s]
[INFO] Ambari Python Client .. SUCCESS [0.618s]
[INFO] Ambari Groovy Client .. SUCCESS [1.972s]
[INFO] Ambari Shell .. SUCCESS [0.044s]
[INFO] Ambari Python Shell ... SUCCESS [0.463s]
[INFO] Ambari Groovy Shell ... SUCCESS [0.830s]
[INFO] ambari-logsearch .. SUCCESS [0.233s]
[INFO] Ambari Logsearch Appender . SUCCESS [0.316s]
[INFO] Ambari Logsearch Solr Client .. SUCCESS [1.024s]
[INFO] Ambari Logsearch Portal ... SUCCESS [5.953s]
[INFO] Ambari Logsearch Log Feeder ... SUCCESS [1.429s]
[INFO] Ambari Logsearch Assembly . SUCCESS [0.047s]
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 8:38.840s
[INFO] Finished at: Wed Jun 15 13:13:20 PDT 2016
[INFO] Final Memory: 206M/1529M
[INFO] 


Thanks,

Aravindan Vijayan



Re: Review Request 48726: AMBARI-17247 : Populate audit to solr / hdfs properties for Atlas

2016-06-15 Thread Srimanth Gunturi

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48726/#review137869
---


Ship it!




Ship It!

- Srimanth Gunturi


On June 15, 2016, 3:42 p.m., Gautam Borad wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48726/
> ---
> 
> (Updated June 15, 2016, 3:42 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Madhan 
> Neethiraj, Srimanth Gunturi, and Velmurugan Periasamy.
> 
> 
> Bugs: AMBARI-17247
> https://issues.apache.org/jira/browse/AMBARI-17247
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> **Issue :**
> If Ranger Atlas Plugin is enabled and audit to solr or hdfs are enabled from 
> Ranger then, need to populate solr / hdfs audit configurations in atlas 
> configs.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/configuration/ranger-atlas-plugin-properties.xml
>  f3bdc2a 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
> 614f0b1 
> 
> Diff: https://reviews.apache.org/r/48726/diff/
> 
> 
> Testing
> ---
> 
> Verified recommendations for audit to solr as well as hdfs for Atlas (if 
> enabled from Ranger configs)
> 
> 
> Thanks,
> 
> Gautam Borad
> 
>



Re: Review Request 48766: MySQL service status needs to be more robust

2016-06-15 Thread Andrew Onischuk

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48766/#review137885
---




ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/mysql_service.py
 (line 49)


We should probably do a checked_call here to give an exception with a good 
information in case we cannot do this.

Also why to we need timeout here?


- Andrew Onischuk


On June 15, 2016, 10:39 p.m., Juanjo  Marron wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48766/
> ---
> 
> (Updated June 15, 2016, 10:39 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Andrew Onischuk, and Dmitro 
> Lisnichenko.
> 
> 
> Bugs: AMBARI-13238
> https://issues.apache.org/jira/browse/AMBARI-13238
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> The MySQL service status in mysql_service.py simply checks for a process with 
> name mysqld. In our environment, a different service ran another MySQL 
> instance on that node and as a result, the status of the MySQL service in 
> Hive showed green (because it could find a mysqld process) even though the 
> instance used by Hive wasn't started. That also made the "Start Service" 
> action for Hive fail, because the metastore service couldn't connect to the 
> MySQL database.
> 
> The proposed fix makes the service check more robust by retrieving the 
> pid_file of the MySQL instance first by running "mysqladmin variables" and 
> parsing out the pid_file. Then it checks if the process exists. 
> 
> A new pacth is proposed based on the reviews added to the original one 
> (https://reviews.apache.org/r/39146/)
> 
> The patch relies now on the pid (similar to other status check). mysql 
> pid_file is not known a priori (it is not a service property similar to other 
> ambari components) and it depends on the OS. 
> The pid file location is obatined with  "mysqladmin variables" command (which 
> seems recognized by all the Ambari supported OS). 
> The Hive MySQL server instance is started without passing in any parameters 
> so It uses the default configuration. Any other MySQL instance that could be 
> running on the machine would need to be started with explicit config.
> Hence just running mysqladmin will return the values for the Hive MySQL 
> instance.
> The pid dile name is parsed by a new method which has been created to obatin 
> it from the mysqladmin variables return.
> The status relies now on check_process_status() wich is already an ambari 
> function.
> Unit Test have been modified accordingly to avoid the pgrep verification that 
> could create conflicts with others mysql instances working on the same node.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/mysql_service.py
>  7862774 
>   ambari-server/src/test/python/stacks/2.0.6/HIVE/test_mysql_server.py 
> 827f6f7 
> 
> Diff: https://reviews.apache.org/r/48766/diff/
> 
> 
> Testing
> ---
> 
> +1 overall. Here are the results of testing the latest attachment 
> http://issues.apache.org/jira/secure/attachment/12810584/AMBARI-13238.patch
> against trunk revision .
> +1 @author. The patch does not contain any @author tags.
> +1 tests included. The patch appears to include 1 new or modified test files.
> +1 javac. The applied patch does not increase the total number of javac 
> compiler warnings.
> +1 release audit. The applied patch does not increase the total number of 
> release audit warnings.
> +1 core tests. The patch passed unit tests in ambari-server.
> Test results: 
> https://builds.apache.org/job/Ambari-trunk-test-patch/7367//testReport/
> Console output: 
> https://builds.apache.org/job/Ambari-trunk-test-patch/7367//console
> This message is automatically generated.
> 
> 
> Thanks,
> 
> Juanjo  Marron
> 
>



Re: Review Request 48739: Remove CascadeType in ServiceComponentHistoryEntity for ServiceComponentDesiredStateEntity

2016-06-15 Thread Alejandro Fernandez

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48739/#review137891
---




ambari-server/src/main/java/org/apache/ambari/server/orm/entities/ServiceComponentHistoryEntity.java
 (line 40)


This is a new table, should we make it consistent, 
service_component_history?


- Alejandro Fernandez


On June 15, 2016, 9:19 p.m., Ajit Kumar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48739/
> ---
> 
> (Updated June 15, 2016, 9:19 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, 
> and Sumit Gupta.
> 
> 
> Bugs: AMBARI-17260
> https://issues.apache.org/jira/browse/AMBARI-17260
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Remove CascadeType in ServiceComponentHistoryEntity for 
> ServiceComponentDesiredStateEntity
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/ServiceComponentHistoryEntity.java
>  e7fef71 
> 
> Diff: https://reviews.apache.org/r/48739/diff/
> 
> 
> Testing
> ---
> 
> Tested manually by deleting service
> 
> 
> Thanks,
> 
> Ajit Kumar
> 
>



Review Request 48766: MySQL service status needs to be more robust

2016-06-15 Thread Juanjo Marron

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48766/
---

Review request for Ambari, Alejandro Fernandez, Andrew Onischuk, and Dmitro 
Lisnichenko.


Bugs: AMBARI-13238
https://issues.apache.org/jira/browse/AMBARI-13238


Repository: ambari


Description
---

The MySQL service status in mysql_service.py simply checks for a process with 
name mysqld. In our environment, a different service ran another MySQL instance 
on that node and as a result, the status of the MySQL service in Hive showed 
green (because it could find a mysqld process) even though the instance used by 
Hive wasn't started. That also made the "Start Service" action for Hive fail, 
because the metastore service couldn't connect to the MySQL database.

The proposed fix makes the service check more robust by retrieving the pid_file 
of the MySQL instance first by running "mysqladmin variables" and parsing out 
the pid_file. Then it checks if the process exists. 

A new pacth is proposed based on the reviews added to the original one 
(https://reviews.apache.org/r/39146/)

The patch relies now on the pid (similar to other status check). mysql pid_file 
is not known a priori (it is not a service property similar to other ambari 
components) and it depends on the OS. 
The pid file location is obatined with  "mysqladmin variables" command (which 
seems recognized by all the Ambari supported OS). 
The Hive MySQL server instance is started without passing in any parameters so 
It uses the default configuration. Any other MySQL instance that could be 
running on the machine would need to be started with explicit config.
Hence just running mysqladmin will return the values for the Hive MySQL 
instance.
The pid dile name is parsed by a new method which has been created to obatin it 
from the mysqladmin variables return.
The status relies now on check_process_status() wich is already an ambari 
function.
Unit Test have been modified accordingly to avoid the pgrep verification that 
could create conflicts with others mysql instances working on the same node.


Diffs
-

  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/mysql_service.py
 7862774 
  ambari-server/src/test/python/stacks/2.0.6/HIVE/test_mysql_server.py 827f6f7 

Diff: https://reviews.apache.org/r/48766/diff/


Testing
---

+1 overall. Here are the results of testing the latest attachment 
http://issues.apache.org/jira/secure/attachment/12810584/AMBARI-13238.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 1 new or modified test files.
+1 javac. The applied patch does not increase the total number of javac 
compiler warnings.
+1 release audit. The applied patch does not increase the total number of 
release audit warnings.
+1 core tests. The patch passed unit tests in ambari-server.
Test results: 
https://builds.apache.org/job/Ambari-trunk-test-patch/7367//testReport/
Console output: 
https://builds.apache.org/job/Ambari-trunk-test-patch/7367//console
This message is automatically generated.


Thanks,

Juanjo  Marron



Re: Review Request 47941: [AMBARI-16920] Follow up issue for Spark2 stack definition

2016-06-15 Thread Jeff Zhang

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47941/
---

(Updated June 15, 2016, 11:06 p.m.)


Review request for Ambari, Jayush Luniya and Sumit Mohanty.


Changes
---

Patch rebased


Bugs: AMBARI-16920
https://issues.apache.org/jira/browse/AMBARI-16920


Repository: ambari


Description
---

This a followup ticket for add spark2 stack definition. There's serveral issues:
1.  Spark2 thrift server can not started due to miss of 
spark-thrift-fairscheduler.xml
2.  Miss of add spark2 cache file in copy_barball.py
3.  Miss the role_commnad_order of spark2
4.  conf_select is missign for spark2


Diffs (updated)
-

  
ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py
 4eb0015 
  
ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py
 286df8d 
  
ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/params.py
 be99edd 
  
ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/spark_service.py
 c2385df 
  ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 
f6011b0 

Diff: https://reviews.apache.org/r/47941/diff/


Testing
---

Manually verified.


Thanks,

Jeff Zhang



Re: Review Request 48766: MySQL service status needs to be more robust

2016-06-15 Thread Juanjo Marron


> On June 15, 2016, 11:44 p.m., Andrew Onischuk wrote:
> > ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/mysql_service.py,
> >  line 54
> > 
> >
> > We should probably do a checked_call here to give an exception with a 
> > good information in case we cannot do this.
> > 
> > Also why to we need timeout here?

Hi Andrew!
Thanks for the quick review

I will remove the timeout. I saw several ps -ef calls fixing the timeout and I 
thought it could be appropiate.

Regarding the check call. I dont know if I followed you , but:
If the shell call  fails in the mysqladmin command, mysql_pid_file will be None 
(meaning mysql service is not running) and check_process_status() will throw 
ComponentIsNotRunning exception as expected
I could capture a log message:
else:
Logger.warning('mysql process not running')

but I think it will be reduntant with the exception.

What do you think?


- Juanjo


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48766/#review137885
---


On June 15, 2016, 10:39 p.m., Juanjo  Marron wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48766/
> ---
> 
> (Updated June 15, 2016, 10:39 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Andrew Onischuk, and Dmitro 
> Lisnichenko.
> 
> 
> Bugs: AMBARI-13238
> https://issues.apache.org/jira/browse/AMBARI-13238
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> The MySQL service status in mysql_service.py simply checks for a process with 
> name mysqld. In our environment, a different service ran another MySQL 
> instance on that node and as a result, the status of the MySQL service in 
> Hive showed green (because it could find a mysqld process) even though the 
> instance used by Hive wasn't started. That also made the "Start Service" 
> action for Hive fail, because the metastore service couldn't connect to the 
> MySQL database.
> 
> The proposed fix makes the service check more robust by retrieving the 
> pid_file of the MySQL instance first by running "mysqladmin variables" and 
> parsing out the pid_file. Then it checks if the process exists. 
> 
> A new pacth is proposed based on the reviews added to the original one 
> (https://reviews.apache.org/r/39146/)
> 
> The patch relies now on the pid (similar to other status check). mysql 
> pid_file is not known a priori (it is not a service property similar to other 
> ambari components) and it depends on the OS. 
> The pid file location is obatined with  "mysqladmin variables" command (which 
> seems recognized by all the Ambari supported OS). 
> The Hive MySQL server instance is started without passing in any parameters 
> so It uses the default configuration. Any other MySQL instance that could be 
> running on the machine would need to be started with explicit config.
> Hence just running mysqladmin will return the values for the Hive MySQL 
> instance.
> The pid dile name is parsed by a new method which has been created to obatin 
> it from the mysqladmin variables return.
> The status relies now on check_process_status() wich is already an ambari 
> function.
> Unit Test have been modified accordingly to avoid the pgrep verification that 
> could create conflicts with others mysql instances working on the same node.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/mysql_service.py
>  7862774 
>   ambari-server/src/test/python/stacks/2.0.6/HIVE/test_mysql_server.py 
> 827f6f7 
> 
> Diff: https://reviews.apache.org/r/48766/diff/
> 
> 
> Testing
> ---
> 
> +1 overall. Here are the results of testing the latest attachment 
> http://issues.apache.org/jira/secure/attachment/12810584/AMBARI-13238.patch
> against trunk revision .
> +1 @author. The patch does not contain any @author tags.
> +1 tests included. The patch appears to include 1 new or modified test files.
> +1 javac. The applied patch does not increase the total number of javac 
> compiler warnings.
> +1 release audit. The applied patch does not increase the total number of 
> release audit warnings.
> +1 core tests. The patch passed unit tests in ambari-server.
> Test results: 
> https://builds.apache.org/job/Ambari-trunk-test-patch/7367//testReport/
> Console output: 
> https://builds.apache.org/job/Ambari-trunk-test-patch/7367//console
> This message is automatically generated.
> 
> 
> Thanks,
> 
> Juanjo  Marron
> 
>



Re: Review Request 48766: MySQL service status needs to be more robust

2016-06-15 Thread Alejandro Fernandez

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48766/#review137881
---


Ship it!




Ship It!

- Alejandro Fernandez


On June 15, 2016, 10:39 p.m., Juanjo  Marron wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48766/
> ---
> 
> (Updated June 15, 2016, 10:39 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Andrew Onischuk, and Dmitro 
> Lisnichenko.
> 
> 
> Bugs: AMBARI-13238
> https://issues.apache.org/jira/browse/AMBARI-13238
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> The MySQL service status in mysql_service.py simply checks for a process with 
> name mysqld. In our environment, a different service ran another MySQL 
> instance on that node and as a result, the status of the MySQL service in 
> Hive showed green (because it could find a mysqld process) even though the 
> instance used by Hive wasn't started. That also made the "Start Service" 
> action for Hive fail, because the metastore service couldn't connect to the 
> MySQL database.
> 
> The proposed fix makes the service check more robust by retrieving the 
> pid_file of the MySQL instance first by running "mysqladmin variables" and 
> parsing out the pid_file. Then it checks if the process exists. 
> 
> A new pacth is proposed based on the reviews added to the original one 
> (https://reviews.apache.org/r/39146/)
> 
> The patch relies now on the pid (similar to other status check). mysql 
> pid_file is not known a priori (it is not a service property similar to other 
> ambari components) and it depends on the OS. 
> The pid file location is obatined with  "mysqladmin variables" command (which 
> seems recognized by all the Ambari supported OS). 
> The Hive MySQL server instance is started without passing in any parameters 
> so It uses the default configuration. Any other MySQL instance that could be 
> running on the machine would need to be started with explicit config.
> Hence just running mysqladmin will return the values for the Hive MySQL 
> instance.
> The pid dile name is parsed by a new method which has been created to obatin 
> it from the mysqladmin variables return.
> The status relies now on check_process_status() wich is already an ambari 
> function.
> Unit Test have been modified accordingly to avoid the pgrep verification that 
> could create conflicts with others mysql instances working on the same node.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/mysql_service.py
>  7862774 
>   ambari-server/src/test/python/stacks/2.0.6/HIVE/test_mysql_server.py 
> 827f6f7 
> 
> Diff: https://reviews.apache.org/r/48766/diff/
> 
> 
> Testing
> ---
> 
> +1 overall. Here are the results of testing the latest attachment 
> http://issues.apache.org/jira/secure/attachment/12810584/AMBARI-13238.patch
> against trunk revision .
> +1 @author. The patch does not contain any @author tags.
> +1 tests included. The patch appears to include 1 new or modified test files.
> +1 javac. The applied patch does not increase the total number of javac 
> compiler warnings.
> +1 release audit. The applied patch does not increase the total number of 
> release audit warnings.
> +1 core tests. The patch passed unit tests in ambari-server.
> Test results: 
> https://builds.apache.org/job/Ambari-trunk-test-patch/7367//testReport/
> Console output: 
> https://builds.apache.org/job/Ambari-trunk-test-patch/7367//console
> This message is automatically generated.
> 
> 
> Thanks,
> 
> Juanjo  Marron
> 
>



Re: Review Request 48766: MySQL service status needs to be more robust

2016-06-15 Thread Juanjo Marron


> On June 15, 2016, 11:44 p.m., Andrew Onischuk wrote:
> > ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/mysql_service.py,
> >  line 54
> > 
> >
> > We should probably do a checked_call here to give an exception with a 
> > good information in case we cannot do this.
> > 
> > Also why to we need timeout here?
> 
> Juanjo  Marron wrote:
> Hi Andrew!
> Thanks for the quick review
> 
> I will remove the timeout. I saw several ps -ef calls fixing the timeout 
> and I thought it could be appropiate.
> 
> Regarding the check call. I dont know if I followed you , but:
> If the shell call  fails in the mysqladmin command, mysql_pid_file will 
> be None (meaning mysql service is not running) and check_process_status() 
> will throw ComponentIsNotRunning exception as expected
> I could capture a log message:
> else:
> Logger.warning('mysql process not running')
> 
> but I think it will be reduntant with the exception.
> 
> What do you think?

Ohh I see, I guess you are refering to shell.checked_call() for testing. I was 
not aware of this resource. (Always learning, sorry)
Any good eaxmple I could follow?

Thanks


- Juanjo


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48766/#review137885
---


On June 15, 2016, 10:39 p.m., Juanjo  Marron wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48766/
> ---
> 
> (Updated June 15, 2016, 10:39 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Andrew Onischuk, and Dmitro 
> Lisnichenko.
> 
> 
> Bugs: AMBARI-13238
> https://issues.apache.org/jira/browse/AMBARI-13238
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> The MySQL service status in mysql_service.py simply checks for a process with 
> name mysqld. In our environment, a different service ran another MySQL 
> instance on that node and as a result, the status of the MySQL service in 
> Hive showed green (because it could find a mysqld process) even though the 
> instance used by Hive wasn't started. That also made the "Start Service" 
> action for Hive fail, because the metastore service couldn't connect to the 
> MySQL database.
> 
> The proposed fix makes the service check more robust by retrieving the 
> pid_file of the MySQL instance first by running "mysqladmin variables" and 
> parsing out the pid_file. Then it checks if the process exists. 
> 
> A new pacth is proposed based on the reviews added to the original one 
> (https://reviews.apache.org/r/39146/)
> 
> The patch relies now on the pid (similar to other status check). mysql 
> pid_file is not known a priori (it is not a service property similar to other 
> ambari components) and it depends on the OS. 
> The pid file location is obatined with  "mysqladmin variables" command (which 
> seems recognized by all the Ambari supported OS). 
> The Hive MySQL server instance is started without passing in any parameters 
> so It uses the default configuration. Any other MySQL instance that could be 
> running on the machine would need to be started with explicit config.
> Hence just running mysqladmin will return the values for the Hive MySQL 
> instance.
> The pid dile name is parsed by a new method which has been created to obatin 
> it from the mysqladmin variables return.
> The status relies now on check_process_status() wich is already an ambari 
> function.
> Unit Test have been modified accordingly to avoid the pgrep verification that 
> could create conflicts with others mysql instances working on the same node.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/mysql_service.py
>  7862774 
>   ambari-server/src/test/python/stacks/2.0.6/HIVE/test_mysql_server.py 
> 827f6f7 
> 
> Diff: https://reviews.apache.org/r/48766/diff/
> 
> 
> Testing
> ---
> 
> +1 overall. Here are the results of testing the latest attachment 
> http://issues.apache.org/jira/secure/attachment/12810584/AMBARI-13238.patch
> against trunk revision .
> +1 @author. The patch does not contain any @author tags.
> +1 tests included. The patch appears to include 1 new or modified test files.
> +1 javac. The applied patch does not increase the total number of javac 
> compiler warnings.
> +1 release audit. The applied patch does not increase the total number of 
> release audit warnings.
> +1 core tests. The patch passed unit tests in ambari-server.
> Test results: 
> https://builds.apache.org/job/Ambari-trunk-test-patch/7367//testReport/
> Console output: 
> 

Review Request 48772: [AMBARI-17243] Use " livy-${cluster-name}@${realm}" instead of " livy@${realm}" for identity "livy.server.kerberos.principal"

2016-06-15 Thread Jeff Zhang

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48772/
---

Review request for Ambari, Jayush Luniya and Sumit Mohanty.


Bugs: AMBARI-17243
https://issues.apache.org/jira/browse/AMBARI-17243


Repository: ambari


Description
---

To have unique principal names that is the convention followed by rest of the 
principals. 
Noticed in stack deploy this principal does not have cluster name.

It is a name convention to include cluster name in principals


Diffs
-

  ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/kerberos.json 
d75364b 

Diff: https://reviews.apache.org/r/48772/diff/


Testing
---

Verify it manually.


Thanks,

Jeff Zhang



Re: Review Request 48607: AMBARI-17181: Add some of value-attributes to property files in AMBARI_METRICS

2016-06-15 Thread Masahiro Tanaka


> On 6月 15, 2016, 5:56 p.m., Aravindan Vijayan wrote:
> > Masahiro, was this patch manually tested on a cluster?

Thank you for reviewing ! I retested on a cluster and added screenshots before 
and after patched.
See https://issues.apache.org/jira/browse/AMBARI-17181
I just checked that the WebUI has changed.

I'm sorry but I noticed that I can't apply the previous patch to the latest 
trunk, I update the patch with some modification.


- Masahiro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48607/#review137783
---


On 6月 16, 2016, 3:55 a.m., Masahiro Tanaka wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48607/
> ---
> 
> (Updated 6月 16, 2016, 3:55 a.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-17181
> https://issues.apache.org/jira/browse/AMBARI-17181
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Some of the property files in AMBARI_METRICS lack value-attributes.
> It would be nice to have value-attributes on most of the properties. If so, 
> we can notice a careless mistakes in Ambari Server WebUI.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-grafana-env.xml
>  eaafc6b 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-grafana-ini.xml
>  da4599e 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-hbase-env.xml
>  3ce1af2 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-hbase-site.xml
>  f4e5fb2 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml
>  6484285 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-ssl-server.xml
>  6f9c6dc 
> 
> Diff: https://reviews.apache.org/r/48607/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Masahiro Tanaka
> 
>