Re: Review Request 51766: AMBARI-18351. Preserve output while executing HSI/LLAP's "run.sh" command for launching LLAP.

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51766/#review148387
---


Ship it!




Ship It!

- Sumit Mohanty


On Sept. 9, 2016, 11:42 p.m., Swapan Shridhar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51766/
> ---
> 
> (Updated Sept. 9, 2016, 11:42 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Andrew Onischuk, and Sumit 
> Mohanty.
> 
> 
> Bugs: AMBARI-18351
> https://issues.apache.org/jira/browse/AMBARI-18351
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> **Issue:**
> 
> Ambari currently runs run.sh for LLAP as such:
> 
> ==
> 2016-08-29 20:34:47,770 - 
> Execute['/var/lib/ambari-agent/tmp/llap-slider2016-08-29_20-34-01/run.sh'] 
> {'user': 'hive'}
> 2016-08-29 20:35:51,856 - Submitted LLAP app name : llap0
> ==
> 
> Above is from Hive Interactive start operation output. Good to have the 
> output (stdout and stderr) of 'run.sh', for debugging purposes.
> 
> **Fix:**
> 
> - Included the output (stdout and stderr) of 'run.sh' command in the 
> operation logs, for debugging purposes.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server_interactive.py
>  2df001c 
>   ambari-server/src/test/python/stacks/2.5/HIVE/test_hive_server_int.py 
> 389e45d 
> 
> Diff: https://reviews.apache.org/r/51766/diff/
> 
> 
> Testing
> ---
> 
> - Python UT passes.
> - Jenkins Pending.
>
> - Tested on live cluster.
> Logs: (o/p of run.sh)
> 
> ==
> 2016-09-09 18:30:11,250 - 
> checked_call['/var/lib/ambari-agent/tmp/llap-slider2016-09-09_18-24-36/run.sh']
>  {'logoutput': True, 'user': 'hive', 'stderr': -1}
> 2016-09-09 18:30:15,750 [main] INFO  impl.TimelineClientImpl - Timeline 
> service address: 
> http://swap-8thsept-hive-2.openstacklocal:8188/ws/v1/timeline/
> 2016-09-09 18:30:17,324 [main] WARN  shortcircuit.DomainSocketFactory - The 
> short-circuit local reads feature cannot be used because libhadoop cannot be 
> loaded.
> 2016-09-09 18:30:17,343 [main] INFO  client.RMProxy - Connecting to 
> ResourceManager at swap-8thsept-hive-2.openstacklocal/172.22.114.139:8050
> 2016-09-09 18:30:17,581 [main] INFO  client.AHSProxy - Connecting to 
> Application History server at 
> swap-8thsept-hive-2.openstacklocal/172.22.114.139:10200
> 2016-09-09 18:30:17,795 [main] ERROR main.ServiceLauncher - Unknown 
> application instance : llap0
>  (definition not found at 
> hdfs://swap-8thsept-hive-1.openstacklocal:8020/user/hive/.slider/cluster/llap0/app_config.json
> 2016-09-09 18:30:17,799 [main] INFO  util.ExitUtil - Exiting with status 69
> 2016-09-09 18:30:23,168 [main] INFO  impl.TimelineClientImpl - Timeline 
> service address: 
> http://swap-8thsept-hive-2.openstacklocal:8188/ws/v1/timeline/
> 2016-09-09 18:30:24,965 [main] WARN  shortcircuit.DomainSocketFactory - The 
> short-circuit local reads feature cannot be used because libhadoop cannot be 
> loaded.
> 2016-09-09 18:30:24,989 [main] INFO  client.RMProxy - Connecting to 
> ResourceManager at swap-8thsept-hive-2.openstacklocal/172.22.114.139:8050
> 2016-09-09 18:30:25,166 [main] INFO  client.AHSProxy - Connecting to 
> Application History server at 
> swap-8thsept-hive-2.openstacklocal/172.22.114.139:10200
> 2016-09-09 18:30:25,793 [main] INFO  imps.CuratorFrameworkImpl - Starting
> 2016-09-09 18:30:25,921 [main-EventThread] INFO  state.ConnectionStateManager 
> - State change: CONNECTED
> 2016-09-09 18:30:25,978 [main] INFO  client.SliderClient - Destroyed cluster 
> llap0
> 2016-09-09 18:30:25,981 [main] INFO  util.ExitUtil - Exiting with status 0
> 2016-09-09 18:30:30,649 [main] INFO  impl.TimelineClientImpl - Timeline 
> service address: 
> http://swap-8thsept-hive-2.openstacklocal:8188/ws/v1/timeline/
> 2016-09-09 18:30:32,060 [main] WARN  shortcircuit.DomainSocketFactory - The 
> short-circuit local reads feature cannot be used because libhadoop cannot be 
> loaded.
> 2016-09-09 18:30:32,081 [main] INFO  client.RMProxy - Connecting to 
> ResourceManager at swap-8thsept-hive-2.openstacklocal/172.22.114.139:8050
> 2016-09-09 18:30:32,237 [main] INFO  client.AHSProxy - Connecting to 
> Application History server at 
> swap-8thsept-hive-2.openstacklocal/172.22.114.139:10200
> 2016-09-09 18:30:32,475 [main] WARN  client.SliderClient - The 
> install-package option has been deprecated. 

Review Request 51766: AMBARI-18351. Preserve output while executing HSI/LLAP's "run.sh" command for launching LLAP.

[Swapan Shridhar]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51766/
---

Review request for Ambari, Alejandro Fernandez, Andrew Onischuk, and Sumit 
Mohanty.


Bugs: AMBARI-18351
https://issues.apache.org/jira/browse/AMBARI-18351


Repository: ambari


Description
---

**Issue:**

Ambari currently runs run.sh for LLAP as such:

==
2016-08-29 20:34:47,770 - 
Execute['/var/lib/ambari-agent/tmp/llap-slider2016-08-29_20-34-01/run.sh'] 
{'user': 'hive'}
2016-08-29 20:35:51,856 - Submitted LLAP app name : llap0
==

Above is from Hive Interactive start operation output. Good to have the output 
(stdout and stderr) of 'run.sh', for debugging purposes.

**Fix:**

- Included the output (stdout and stderr) of 'run.sh' command in the operation 
logs, for debugging purposes.


Diffs
-

  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server_interactive.py
 2df001c 
  ambari-server/src/test/python/stacks/2.5/HIVE/test_hive_server_int.py 389e45d 

Diff: https://reviews.apache.org/r/51766/diff/


Testing
---

- Python UT passes.
- Jenkins Pending.
   
- Tested on live cluster.
Logs: (o/p of run.sh)

==
2016-09-09 18:30:11,250 - 
checked_call['/var/lib/ambari-agent/tmp/llap-slider2016-09-09_18-24-36/run.sh'] 
{'logoutput': True, 'user': 'hive', 'stderr': -1}
2016-09-09 18:30:15,750 [main] INFO  impl.TimelineClientImpl - Timeline service 
address: http://swap-8thsept-hive-2.openstacklocal:8188/ws/v1/timeline/
2016-09-09 18:30:17,324 [main] WARN  shortcircuit.DomainSocketFactory - The 
short-circuit local reads feature cannot be used because libhadoop cannot be 
loaded.
2016-09-09 18:30:17,343 [main] INFO  client.RMProxy - Connecting to 
ResourceManager at swap-8thsept-hive-2.openstacklocal/172.22.114.139:8050
2016-09-09 18:30:17,581 [main] INFO  client.AHSProxy - Connecting to 
Application History server at 
swap-8thsept-hive-2.openstacklocal/172.22.114.139:10200
2016-09-09 18:30:17,795 [main] ERROR main.ServiceLauncher - Unknown application 
instance : llap0
 (definition not found at 
hdfs://swap-8thsept-hive-1.openstacklocal:8020/user/hive/.slider/cluster/llap0/app_config.json
2016-09-09 18:30:17,799 [main] INFO  util.ExitUtil - Exiting with status 69
2016-09-09 18:30:23,168 [main] INFO  impl.TimelineClientImpl - Timeline service 
address: http://swap-8thsept-hive-2.openstacklocal:8188/ws/v1/timeline/
2016-09-09 18:30:24,965 [main] WARN  shortcircuit.DomainSocketFactory - The 
short-circuit local reads feature cannot be used because libhadoop cannot be 
loaded.
2016-09-09 18:30:24,989 [main] INFO  client.RMProxy - Connecting to 
ResourceManager at swap-8thsept-hive-2.openstacklocal/172.22.114.139:8050
2016-09-09 18:30:25,166 [main] INFO  client.AHSProxy - Connecting to 
Application History server at 
swap-8thsept-hive-2.openstacklocal/172.22.114.139:10200
2016-09-09 18:30:25,793 [main] INFO  imps.CuratorFrameworkImpl - Starting
2016-09-09 18:30:25,921 [main-EventThread] INFO  state.ConnectionStateManager - 
State change: CONNECTED
2016-09-09 18:30:25,978 [main] INFO  client.SliderClient - Destroyed cluster 
llap0
2016-09-09 18:30:25,981 [main] INFO  util.ExitUtil - Exiting with status 0
2016-09-09 18:30:30,649 [main] INFO  impl.TimelineClientImpl - Timeline service 
address: http://swap-8thsept-hive-2.openstacklocal:8188/ws/v1/timeline/
2016-09-09 18:30:32,060 [main] WARN  shortcircuit.DomainSocketFactory - The 
short-circuit local reads feature cannot be used because libhadoop cannot be 
loaded.
2016-09-09 18:30:32,081 [main] INFO  client.RMProxy - Connecting to 
ResourceManager at swap-8thsept-hive-2.openstacklocal/172.22.114.139:8050
2016-09-09 18:30:32,237 [main] INFO  client.AHSProxy - Connecting to 
Application History server at 
swap-8thsept-hive-2.openstacklocal/172.22.114.139:10200
2016-09-09 18:30:32,475 [main] WARN  client.SliderClient - The install-package 
option has been deprecated. Please use 'package --install'.
2016-09-09 18:30:32,505 [main] INFO  client.SliderClient - Installing package 
file:/var/lib/ambari-agent/tmp/llap-slider2016-09-09_18-24-36/llap-09Sep2016.zip
 at 
hdfs://swap-8thsept-hive-1.openstacklocal:8020/user/hive/.slider/package/LLAP/llap-09Sep2016.zip
 and overwrite is true.
2016-09-09 18:30:35,399 [main] INFO  util.ExitUtil - Exiting with status 0
2016-09-09 18:30:39,883 [main] INFO  impl.TimelineClientImpl - Timeline service 
address: http://swap-8thsept-hive-2.openstacklocal:8188/ws/v1/timeline/
2016-09-09 18:30:41,393 [main] WARN  

Re: Review Request 51724: Add Kerberos HTTP SPNEGO authentication support to Ambari Metrics Monitor

[Qin Liu]

> On Sept. 8, 2016, 6:13 p.m., Sid Wagle wrote:
> > ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/emitter.py,
> >  line 145
> > 
> >
> > Shouldn't this be checking for None instead of empty string ?
> 
> Qin Liu wrote:
> I will do it. Thanks Sid!

Sorry, I was looking at the wrong place. It should be checking for empty string 
instead of None because the key "Negotiate" is in auth_header.


- Qin


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51724/#review148224
---


On Sept. 8, 2016, 5:27 p.m., Qin Liu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51724/
> ---
> 
> (Updated Sept. 8, 2016, 5:27 p.m.)
> 
> 
> Review request for Ambari, Di Li, Dmytro Sen, and Sid Wagle.
> 
> 
> Bugs: AMBARI-17898
> https://issues.apache.org/jira/browse/AMBARI-17898
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> ()This is a subtask of AMBARI-14384 "Ambari Metrics doesn't use SPNEGO to 
> authenticate".
> 
> In a Kerberos enabled cluster with SPNEGO enabled on Hadoop APIs, Ambari 
> Metrics Collector (in AMS distributed mode) web-console will be Kerberos HTTP 
> SPNEGO enabled too. But Ambari Metrics Monitor, a client of Ambari Metrics 
> Collector, currently does not support Kerberos HTTP SPNEGO authentication.
> 
> /var/log/ambari-metrics-monitor/ambari-metrics-monitor.out:
> 2015-12-15 13:26:30,663 [INFO] emitter.py:101 - server: 
> http://metrics-collector:6188/ws/v1/timeline/metrics
> 2015-12-15 13:26:30,671 [WARNING] emitter.py:84 - Error sending metrics to 
> server. HTTP Error 401: Authentication required
> 2015-12-15 13:26:30,671 [WARNING] emitter.py:90 - Retrying after 5 ...
> 
> 
> Diffs
> -
> 
>   ambari-metrics/ambari-metrics-host-monitoring/conf/unix/metric_monitor.ini 
> 3e5d861 
>   
> ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/config_reader.py
>  02f0ce3 
>   
> ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/emitter.py 
> 6997108 
>   
> ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/faked_kerberos.py
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/spnego_kerberos_auth.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py
>  fc95aa7 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/templates/metric_monitor.ini.j2
>  383a0de 
> 
> Diff: https://reviews.apache.org/r/51724/diff/
> 
> 
> Testing
> ---
> 
> 1. The fix has been tested with HDP-2.5.0.0/branch-2.4 UI (spnego enabled and 
> non-kerbero scenarios)
> - the patch can be applied to branch-2.4 without any changes. 
> - I was not able to test it on trunk because the current trunk version of 
> metrics collector was not able to start  
> 2. The fix has passed existing unittest cases in both trunk and branch-2.4
> 3. I didn't add new unittest cases because I don't know how to create a 
> kerberos enabled the cluster in unittest cases. - please advise me on this, 
> thx!
> 
> 
> Thanks,
> 
> Qin Liu
> 
>

Re: Review Request 51540: Expose Disabling of Alert Targets in Web Client

[Vivek Ratnavel Subramanian]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51540/
---

(Updated Sept. 9, 2016, 10:07 p.m.)


Review request for Ambari, Jaimin Jetly, Zhe (Joe) Wang, and Yusaku Sako.


Changes
---

Changes made to fix AMBARI-18306.
Verified manually.
Ambari-web unit tests pass
30182 tests complete (26 seconds)
151 tests pending


Bugs: Ambari-18281
https://issues.apache.org/jira/browse/Ambari-18281


Repository: ambari


Description
---

Add an option to Enable/ Disable alert notifications in Manage Notifications 
popup.


Diffs (updated)
-

  
ambari-web/app/controllers/main/alerts/alert_definitions_actions_controller.js 
f3f0387 
  ambari-web/app/styles/modal_popups.less a2343e0 

Diff: https://reviews.apache.org/r/51540/diff/


Testing
---

Verified manually.
Ambari-web unit tests pass.
30014 tests complete (26 seconds)
154 tests pending


File Attachments


v1
  
https://reviews.apache.org/media/uploaded/files/2016/08/31/ad971a3a-3430-4caa-9e7f-df60cc75872c__AMBARI-18281.v1.patch


Thanks,

Vivek Ratnavel Subramanian

Re: Review Request 51744: SmartSense Mandatory Install

[Alejandro Fernandez]

> On Sept. 9, 2016, 9:16 p.m., Alejandro Fernandez wrote:
> > ambari-web/app/controllers/wizard/step4_controller.js, line 37
> > 
> >
> > -1 Why are we requiring HDP stacks to forcibly install SmartSense?
> > This feels like a hidden tactic. Any user that wants to install 
> > SmartSense will add it, just as they do other components; plus, there's 
> > already a dialog box warning if the user doesn't select it.
> > 
> > Devs, Sandbox, etc., don't need it.

UI shouldn't have knowledge about SmartSense. Instead, consider moving this to 
a stack config that is specific to this service.


- Alejandro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51744/#review148373
---


On Sept. 8, 2016, 8:09 p.m., Vivek Ratnavel Subramanian wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51744/
> ---
> 
> (Updated Sept. 8, 2016, 8:09 p.m.)
> 
> 
> Review request for Ambari, Jaimin Jetly, Zhe (Joe) Wang, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-18346
> https://issues.apache.org/jira/browse/AMBARI-18346
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Perform a UI hack to force selection of SmartSense for HDP stacks (the user 
> cannot deselect).
> 
> 
> Diffs
> -
> 
>   ambari-web/app/controllers/wizard/step4_controller.js 04e6784 
>   ambari-web/app/mappers/stack_service_mapper.js c800e83 
>   ambari-web/app/models/stack_service.js 9cf9d49 
>   ambari-web/app/templates/wizard/step4.hbs c62d369 
> 
> Diff: https://reviews.apache.org/r/51744/diff/
> 
> 
> Testing
> ---
> 
> Verified manually.
> Ambari-web unit test pass.
> 29244 tests complete (26 seconds)
> 154 tests pending
> 
> 
> Thanks,
> 
> Vivek Ratnavel Subramanian
> 
>

Re: Review Request 51724: Add Kerberos HTTP SPNEGO authentication support to Ambari Metrics Monitor

[Qin Liu]

> On Sept. 8, 2016, 6:13 p.m., Sid Wagle wrote:
> > ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/spnego_kerberos_auth.py,
> >  line 25
> > 
> >
> > I do not see a "import kerberos" anywhere in Ambari deployed cluster 
> > under: /usr/lib/python2.6/site-packages/
> > 
> > Where does this dependency get fulfiled ?

It is in /usr/lib/python2.6/site-packages/ipalib/plugins on all of my clusters. 
I will include ipa-python in ambari-metrics-assembly/pom.xml. Thanks Sid!


> On Sept. 8, 2016, 6:13 p.m., Sid Wagle wrote:
> > ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/emitter.py,
> >  line 145
> > 
> >
> > Shouldn't this be checking for None instead of empty string ?

I will do it. Thanks Sid!


On Sept. 8, 2016, 6:13 p.m., Qin Liu wrote:
> > We don't have unit test whcih setup secure cluster so we should definitely 
> > test this on trunk build, which should be working now.
> > The changes overall look good just want to make sure we have done some 
> > manual testing before commit.

Hi Sid,
I tried to test it on HDP2.5.0.0/trunk. but I was not able to bring the metrics 
up. Do you have this experience? 
But I will try to test it on HDP2.4/trunk.


- Qin


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51724/#review148224
---


On Sept. 8, 2016, 5:27 p.m., Qin Liu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51724/
> ---
> 
> (Updated Sept. 8, 2016, 5:27 p.m.)
> 
> 
> Review request for Ambari, Di Li, Dmytro Sen, and Sid Wagle.
> 
> 
> Bugs: AMBARI-17898
> https://issues.apache.org/jira/browse/AMBARI-17898
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> ()This is a subtask of AMBARI-14384 "Ambari Metrics doesn't use SPNEGO to 
> authenticate".
> 
> In a Kerberos enabled cluster with SPNEGO enabled on Hadoop APIs, Ambari 
> Metrics Collector (in AMS distributed mode) web-console will be Kerberos HTTP 
> SPNEGO enabled too. But Ambari Metrics Monitor, a client of Ambari Metrics 
> Collector, currently does not support Kerberos HTTP SPNEGO authentication.
> 
> /var/log/ambari-metrics-monitor/ambari-metrics-monitor.out:
> 2015-12-15 13:26:30,663 [INFO] emitter.py:101 - server: 
> http://metrics-collector:6188/ws/v1/timeline/metrics
> 2015-12-15 13:26:30,671 [WARNING] emitter.py:84 - Error sending metrics to 
> server. HTTP Error 401: Authentication required
> 2015-12-15 13:26:30,671 [WARNING] emitter.py:90 - Retrying after 5 ...
> 
> 
> Diffs
> -
> 
>   ambari-metrics/ambari-metrics-host-monitoring/conf/unix/metric_monitor.ini 
> 3e5d861 
>   
> ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/config_reader.py
>  02f0ce3 
>   
> ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/emitter.py 
> 6997108 
>   
> ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/faked_kerberos.py
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/spnego_kerberos_auth.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py
>  fc95aa7 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/templates/metric_monitor.ini.j2
>  383a0de 
> 
> Diff: https://reviews.apache.org/r/51724/diff/
> 
> 
> Testing
> ---
> 
> 1. The fix has been tested with HDP-2.5.0.0/branch-2.4 UI (spnego enabled and 
> non-kerbero scenarios)
> - the patch can be applied to branch-2.4 without any changes. 
> - I was not able to test it on trunk because the current trunk version of 
> metrics collector was not able to start  
> 2. The fix has passed existing unittest cases in both trunk and branch-2.4
> 3. I didn't add new unittest cases because I don't know how to create a 
> kerberos enabled the cluster in unittest cases. - please advise me on this, 
> thx!
> 
> 
> Thanks,
> 
> Qin Liu
> 
>

Re: Review Request 51744: SmartSense Mandatory Install

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51744/#review148373
---




ambari-web/app/controllers/wizard/step4_controller.js (line 37)


-1 Why are we requiring HDP stacks to forcibly install SmartSense?
This feels like a hidden tactic. Any user that wants to install SmartSense 
will add it, just as they do other components; plus, there's already a dialog 
box warning if the user doesn't select it.

Devs, Sandbox, etc., don't need it.


- Alejandro Fernandez


On Sept. 8, 2016, 8:09 p.m., Vivek Ratnavel Subramanian wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51744/
> ---
> 
> (Updated Sept. 8, 2016, 8:09 p.m.)
> 
> 
> Review request for Ambari, Jaimin Jetly, Zhe (Joe) Wang, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-18346
> https://issues.apache.org/jira/browse/AMBARI-18346
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Perform a UI hack to force selection of SmartSense for HDP stacks (the user 
> cannot deselect).
> 
> 
> Diffs
> -
> 
>   ambari-web/app/controllers/wizard/step4_controller.js 04e6784 
>   ambari-web/app/mappers/stack_service_mapper.js c800e83 
>   ambari-web/app/models/stack_service.js 9cf9d49 
>   ambari-web/app/templates/wizard/step4.hbs c62d369 
> 
> Diff: https://reviews.apache.org/r/51744/diff/
> 
> 
> Testing
> ---
> 
> Verified manually.
> Ambari-web unit test pass.
> 29244 tests complete (26 seconds)
> 154 tests pending
> 
> 
> Thanks,
> 
> Vivek Ratnavel Subramanian
> 
>

Re: Review Request 51744: SmartSense Mandatory Install

[Zhe (Joe) Wang]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51744/#review148372
---




ambari-web/app/mappers/stack_service_mapper.js (line 122)


Since the logic here is that if this variable is true, that means this 
service is either disabled or it's installed, using the name "isDisabled" will 
mask partial fact. In another way to sayit, people may be confused about why a 
service is "disabled" when it's just acctually installed.
So maybe call it something like "installedOrDisabled"?


- Zhe (Joe) Wang


On Sept. 8, 2016, 8:09 p.m., Vivek Ratnavel Subramanian wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51744/
> ---
> 
> (Updated Sept. 8, 2016, 8:09 p.m.)
> 
> 
> Review request for Ambari, Jaimin Jetly, Zhe (Joe) Wang, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-18346
> https://issues.apache.org/jira/browse/AMBARI-18346
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Perform a UI hack to force selection of SmartSense for HDP stacks (the user 
> cannot deselect).
> 
> 
> Diffs
> -
> 
>   ambari-web/app/controllers/wizard/step4_controller.js 04e6784 
>   ambari-web/app/mappers/stack_service_mapper.js c800e83 
>   ambari-web/app/models/stack_service.js 9cf9d49 
>   ambari-web/app/templates/wizard/step4.hbs c62d369 
> 
> Diff: https://reviews.apache.org/r/51744/diff/
> 
> 
> Testing
> ---
> 
> Verified manually.
> Ambari-web unit test pass.
> 29244 tests complete (26 seconds)
> 154 tests pending
> 
> 
> Thanks,
> 
> Vivek Ratnavel Subramanian
> 
>

Re: Review Request 51754: AMBARI-18337 Syntax Error in Ambari HAWQ Unit test with Python 2.6

[Lav Jain]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51754/#review148370
---


Ship it!




Ship It!

- Lav Jain


On Sept. 9, 2016, 1:21 a.m., Masahiro Tanaka wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51754/
> ---
> 
> (Updated Sept. 9, 2016, 1:21 a.m.)
> 
> 
> Review request for Ambari, Alexander Denissov, Lav Jain, and Matt.
> 
> 
> Bugs: AMBARI-18337
> https://issues.apache.org/jira/browse/AMBARI-18337
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> With Python 2.6, ` {"A", "B"}  ` syntax isn't allowed.
> 
> Ref. https://docs.python.org/2/library/stdtypes.html#set-types-set-frozenset
> 
> ```
> As of Python 2.7, non-empty sets (not frozensets) can be created by placing a 
> comma-separated list of elements within braces, for example: {'jack', 
> 'sjoerd'}, in addition to the set constructor.
> ```
> 
> Error
> 
> ```
> Traceback (most recent call last):
>   File "unitTests.py", line 129, in stack_test_executor
> modules]
>   File "/usr/lib64/python2.6/unittest.py", line 575, in loadTestsFromName
> module = __import__('.'.join(parts_copy))
>   File 
> "/tmp/ambari/ambari-server/src/test/python/common-services/HAWQ/test_service_advisor.py",
>  li
> ne 646
> self.assertFalse({'HAWQMASTER', 'HAWQSTANDBY'}.issubset(hostComponents))
>   ^
> SyntaxError: invalid syntax
> ```
> 
> 
> Diffs
> -
> 
>   ambari-server/src/test/python/common-services/HAWQ/test_service_advisor.py 
> 71c109f 
> 
> Diff: https://reviews.apache.org/r/51754/diff/
> 
> 
> Testing
> ---
> 
> +1 overall. Here are the results of testing the latest attachment
> http://issues.apache.org/jira/secure/attachment/12827665/AMBARI-18337.patch
> against trunk revision .
> 
> +1 @author. The patch does not contain any @author tags.
> 
> +1 tests included. The patch appears to include 1 new or modified test files.
> 
> +1 javac. The applied patch does not increase the total number of javac 
> compiler warnings.
> 
> +1 release audit. The applied patch does not increase the total number of 
> release audit warnings.
> 
> +1 core tests. The patch passed unit tests in ambari-server.
> 
> Test results: 
> https://builds.apache.org/job/Ambari-trunk-test-patch/8623//testReport/
> Console output: 
> https://builds.apache.org/job/Ambari-trunk-test-patch/8623//console
> 
> This message is automatically generated.
> 
> 
> Thanks,
> 
> Masahiro Tanaka
> 
>

Re: Review Request 51578: AMBARI-18071: Ambari Files View needs to have ability to load security configurations

[Keta Patel]

> On Sept. 8, 2016, 2:41 p.m., Di Li wrote:
> > Ship It!

Thank you Di!


- Keta


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51578/#review148187
---


On Sept. 7, 2016, 4:57 p.m., Keta Patel wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51578/
> ---
> 
> (Updated Sept. 7, 2016, 4:57 p.m.)
> 
> 
> Review request for Ambari, Di Li, DIPAYAN BHOWMICK, and Pallav Kulshreshtha.
> 
> 
> Bugs: AMBARI-18071
> https://issues.apache.org/jira/browse/AMBARI-18071
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When HDFS is configured with Encryption Zones, Files View to browse files 
> will give "No KeyProvider" error.
> 
> Steps to reproduce this issue:
> 1. Configure an encrypted zone in HDFS (Transparent Data Encryption). I have 
> used Hadoop's KMS (installed tar manually).
> 2. Create a Files View instance and provide a user/group with the privilege 
> to use the instance.
> 3. Log into the Ambari console as the user with the Files View permission.
> 4. Open the Files View instance.
> 5. Go to the folder which is configured as an encrypted zone.
> 6. Try to open an existing file in this folder.
> 7. This throws an error - java.io.IOException: No KeyProvider is configured, 
> cannot access an encrypted file. 
> 8. When trying through the shell, opening this file works.
> 
> This happens because Files View doesn't have enough configuration set to 
> browse secured zone. Files view doesn't even provide an option to add this 
> configuration. This is why we see errors "No KeyProvider is configured, 
> cannot access an encrypted file". 
> To work around this, you could download client configuration from HDFS 
> service tab, and copy the core-site.xml and hdfs-site.xml files to 
> /etc/ambari-server/conf, then restart ambari-server. After this, the user is 
> able to open the file in the encrypted zone. Basically, the property 
> "dfs.encryption.key.provider.uri" which provides details of the KeyProvider, 
> is obtained from the hdfs-site.xml present in the /etc/ambari-server/conf 
> folder.
> 
> 
> Diffs
> -
> 
>   
> contrib/views/utils/src/main/java/org/apache/ambari/view/utils/hdfs/ConfigurationBuilder.java
>  d505e9a 
>   
> contrib/views/utils/src/test/java/org/apache/ambari/view/utils/hdfs/ConfigurationBuilderTest.java
>  ed3025b 
> 
> Diff: https://reviews.apache.org/r/51578/diff/
> 
> 
> Testing
> ---
> 
> **CAUSE:**
> ==
> The error of "No KeyProvider is configured" is seen only for those cases when 
> the HDFS uses DistributedFileSystem for its communication. When HDFS uses 
> WebHDFSFileSystem for communication, this error is not seen and the Ambari 
> View instance is able to open the files in the encrypted zones.
> 
> Why Ambari Views use either Distributed or WebHDFS file systems is explained 
> below:
> Ambari views can be created using one of the 3 modes of configuration:
> 1. Local cluster
> 2. Remote cluster
> 3. Custom configuration (no cluster is associated here).
> 
> The HDFS works through abstraction. For Ambari Views, the actual file system 
> used during execution depends on whether the view instance was created using 
> a Local/Remote cluster or using Custom configuration. For instances created 
> using Local/Remote cluster, HDFS uses Distributed File System and for 
> instances created using Custom configuration, HDFS uses WebHDFSFileSystem.
> WebHDFSFileSystem is an integrated part of the HDFS ecosystem. It is aware of 
> all the HDFS configuration. For this reason, when a KMS is configured in 
> HDFS, WebHDFSFileSystem is aware of the KeyProvider and no special config 
> mapping is needed. Thus, even the view instance created using Custom 
> configuration doesn't need any special configuration and can talk to the 
> Encryption Zones successfully.
> 
> However, for view instances created using Local/Remote cluster configuration, 
> HDFS uses the Distributed FileSystem. This Distributed FileSystem works as an 
> HDFS client and hence, is not fully aware of all the HDFS configuration. We 
> need to explicitly provide HDFS properties like 
> "dfs.encryption.key.provider.uri" to these ambari view instances to provide 
> details of the KeyProvider. The proposed fix helps in providing this property 
> value to the view as follows.
> 
> **FIX:**
> 
> The proposed fix (attached as "AMBARI-18071.patch") checks if the current 
> view instance configuration has any cluster associated in its context. If 
> there is an associated cluster then the instance has a Local/Remote cluster 
> configuration and needs to be provided with the HDFS KeyProvider information. 
> Otherwise, the WebHDFSFileSystem will take care of the 

Re: Review Request 51713: After upgrading cluster from HDP-2.4.x to HDP-2.5.x and added atlas service - missing kafka security properties

[Robert Levas]

> On Sept. 8, 2016, 11:13 a.m., Nate Cole wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml,
> >  line 473
> > 
> >
> > If you use KERBEROS and KERBEROS_SERVER as service/component then this 
> > group should get skipped.
> 
> Robert Levas wrote:
> Kerberos can be enabled and not require the KERBEROS services and the 
> KERBEROS_CLIENT to be installed.  In the manual case, they are not since we 
> do not need to distribute keytab files.  Is there a way to condtionally 
> execute an action based on a config property (`cluster-env/security_enabled`) 
> or the cluster's security state.  This would only be to prevent the task from 
> showing up in the list of tasks shown during the upgrade process.  
> Internally, if there is no user-defined Kerberos descriptor, nothing happens 
> - but I can fix the log messages to clarifiy this.
> 
> Nate Cole wrote:
> Hmm, I see what you're saying.  Can you file a future jira to add a way 
> to compute whether an execute-stage should be included, other than the simple 
> service/component check?

Created [AMBARI-18352](https://issues.apache.org/jira/browse/AMBARI-18352) to 
address this


- Robert


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51713/#review148190
---


On Sept. 9, 2016, 2:29 p.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51713/
> ---
> 
> (Updated Sept. 9, 2016, 2:29 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, 
> and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-18335
> https://issues.apache.org/jira/browse/AMBARI-18335
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Steps to repro:
> - Install Ambari 2.2.2
> - Install HDP-2.4.x cluster with Atlas
> - Stop Atlas
> - Upgrade Ambari to 2.4
> - Delete Atlas service
> - Upgrade the cluster to HDP-2.5.x cluster
> - Add Atlas service.
> 
> *Below config properties are missing from atlas-applicataion.properties file 
> for Atlas, Storm, Falcon, Hive services.*
> ```
> atlas.jaas.KafkaClient.option.keyTab = 
> /etc/security/keytabs/atlas.service.keytab
> atlas.jaas.KafkaClient.option.principal = atlas/_h...@example.com
> ```
> 
> From HDP 2.4 to 2.5, the kerberos.json file for Atlas changed.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/UpgradeUserKerberosDescriptor.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptor.java
>  84a9111 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptorContainer.java
>  39ebdaf 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptor.java
>  3cdd9908 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptor.java
>  484f65c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelper.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptor.java
>  7ce1c9f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptor.java
>  0156e4a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosServiceDescriptor.java
>  72dbcfe 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/kerberos.json
>  d569447 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml
>  ad9fc97 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> 27f2010 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml
>  4df5fef 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> b8c51f5 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HBASE/kerberos.json 
> 501bcd3 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.5.xml
>  0b6f762 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.5.xml 
> 49e9d87 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterKerberosDescriptorResourceProviderTest.java
>  f551b42 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosComponentDescriptorTest.java
>  b74f417 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptorTest.java
>  6017fae 
>   
> 

Re: Review Request 51713: After upgrading cluster from HDP-2.4.x to HDP-2.5.x and added atlas service - missing kafka security properties

[Robert Levas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51713/
---

(Updated Sept. 9, 2016, 2:29 p.m.)


Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, and 
Sumit Mohanty.


Bugs: AMBARI-18335
https://issues.apache.org/jira/browse/AMBARI-18335


Repository: ambari


Description
---

Steps to repro:
- Install Ambari 2.2.2
- Install HDP-2.4.x cluster with Atlas
- Stop Atlas
- Upgrade Ambari to 2.4
- Delete Atlas service
- Upgrade the cluster to HDP-2.5.x cluster
- Add Atlas service.

*Below config properties are missing from atlas-applicataion.properties file 
for Atlas, Storm, Falcon, Hive services.*
```
atlas.jaas.KafkaClient.option.keyTab = 
/etc/security/keytabs/atlas.service.keytab
atlas.jaas.KafkaClient.option.principal = atlas/_h...@example.com
```

>From HDP 2.4 to 2.5, the kerberos.json file for Atlas changed.


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/UpgradeUserKerberosDescriptor.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptor.java
 84a9111 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptorContainer.java
 39ebdaf 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptor.java
 3cdd9908 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptor.java
 484f65c 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelper.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptor.java
 7ce1c9f 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptor.java
 0156e4a 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosServiceDescriptor.java
 72dbcfe 
  
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/kerberos.json 
d569447 
  
ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml
 ad9fc97 
  ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
27f2010 
  
ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml
 4df5fef 
  ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
b8c51f5 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/HBASE/kerberos.json 
501bcd3 
  
ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.5.xml
 0b6f762 
  ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.5.xml 
49e9d87 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterKerberosDescriptorResourceProviderTest.java
 f551b42 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosComponentDescriptorTest.java
 b74f417 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptorTest.java
 6017fae 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorTest.java
 9463749 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelperTest.java
 PRE-CREATION 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosIdentityDescriptorTest.java
 874da31 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptorTest.java
 c10d106 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptorTest.java
 5c249e2 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosServiceDescriptorTest.java
 e1af515 

Diff: https://reviews.apache.org/r/51713/diff/


Testing
---

unit tests, manually upgrades

# Local test results: 

[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 1:20:19.796s
[INFO] Finished at: Fri Sep 09 09:29:44 EDT 2016
[INFO] Final Memory: 62M/1882M
[INFO] 

# Jenkins test results: PENDING


Thanks,

Robert Levas

Re: Review Request 51761: AMBARI-18050 - Upgrade pre-req check code needs to be decoupled from CheckDescription class

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51761/#review148350
---


Ship it!




Ship It!

- Alejandro Fernandez


On Sept. 9, 2016, 4:59 p.m., Tim Thorpe wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51761/
> ---
> 
> (Updated Sept. 9, 2016, 4:59 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Di Li, Jayush Luniya, and 
> Nate Cole.
> 
> 
> Bugs: AMBARI-18050
> https://issues.apache.org/jira/browse/AMBARI-18050
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Currently each pre-req check must add an entry in the CheckDescription enum. 
> This limits the ability for third party stacks and extensions to provide 
> their own pre-req checks.
> 
> The CheckDescription enum should be rewritten as a class and each pre-req 
> check class should create an instance of it. This will allow stacks and 
> extensions to include their own pre-req checks in separate jar files without 
> requiring changes to ambari-server java code.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/CheckDescription.java
>  aa8e20c 
> 
> Diff: https://reviews.apache.org/r/51761/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test 
> -Dtest=AbstractCheckDescriptorTest,UpgradeCheckOrderTest,CheckHelperTest
> Ran 267 tests in 8.183s
> Total run:1124
> Total errors:0
> Total failures:0
> 
> 
> Thanks,
> 
> Tim Thorpe
> 
>

Review Request 51762: Declare URL uniqueness in repoinfo.xml

[Dmitro Lisnichenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51762/
---

Review request for Ambari, Jonathan Hurley and Nate Cole.


Bugs: AMBARI-18350
https://issues.apache.org/jira/browse/AMBARI-18350


Repository: ambari


Description
---

Repository info for a stack can contain any number of repo definitions.  For 
HDP, there are two repos used (HDP and HDP-UTILS).  When checking repositories 
for duplicate URL, the code does this:
{noformat}
if (! repositoryEntity.getRepositoryId().startsWith("HDP-UTILS") &&  // 
HDP-UTILS is shared between repo versions
! existingRepoVersion.getId().equals(repositoryVersion.getId())) { // Allow 
modifying already defined repo version
existingRepoUrls.add(repositoryEntity.getBaseUrl());
}
{noformat}

We should make the unique declaration directly for the repository so authors 
don't have to guess.  This entails:
* Change {{repoinfo.xml}} to support a {{}} element within {{}}
* {{}} should apply to, say, versions of HDP that change with each 
release.  Absence of said element implies {{false}} for backward compatibility.
* Change {{RepositoryVersionResourceProvider.validateRepositoryVersion(...)}} 
to use this boolean for uniqueness check.
* Update the {{version_definition.xsd}} to include this new element.


Diffs
-

  ambari-funtest/src/test/resources/stacks/HDP/0.1/repos/repoinfo.xml 1ea1809 
  ambari-funtest/src/test/resources/stacks/HDP/0.2/repos/repoinfo.xml c0095b5 
  ambari-funtest/src/test/resources/stacks/HDP/1.2.0/repos/repoinfo.xml 04613ad 
  ambari-funtest/src/test/resources/stacks/HDP/1.3.0/repos/repoinfo.xml ec0be8f 
  ambari-funtest/src/test/resources/stacks/HDP/1.3.1/repos/repoinfo.xml ed1aed8 
  ambari-funtest/src/test/resources/stacks/HDP/2.0.1/repos/repoinfo.xml eb026a7 
  ambari-funtest/src/test/resources/stacks/HDP/2.0.5/repos/repoinfo.xml 91f0f02 
  ambari-funtest/src/test/resources/stacks/HDP/2.0.6.1/repos/repoinfo.xml 
2739c5c 
  ambari-funtest/src/test/resources/stacks/HDP/2.0.6/repos/repoinfo.xml 2739c5c 
  ambari-funtest/src/test/resources/stacks/HDP/2.0.7/repos/repoinfo.xml 8ebea21 
  
ambari-funtest/src/test/resources/stacks/HDP/2.0.7/services/HBASE/metainfo.xml 
e3ebea1 
  ambari-funtest/src/test/resources/stacks/HDP/2.0.8/repos/repoinfo.xml f8542d0 
  ambari-funtest/src/test/resources/stacks/HDP/2.1.1/repos/repoinfo.xml 9d8a232 
  ambari-funtest/src/test/resources/stacks/HDP/2.2.0.ECS/repos/repoinfo.xml 
6c2f99a 
  ambari-funtest/src/test/resources/stacks/HDP/2.2.0/repos/repoinfo.xml 2a939f1 
  ambari-funtest/src/test/resources/stacks/OTHER/1.0/repos/repoinfo.xml 9d8a232 
  ambari-funtest/src/test/resources/stacks/OTHER/2.0/repos/repoinfo.xml 9d8a232 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/RepositoryResponse.java
 30bd0db 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RepositoryResourceProvider.java
 3203539 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RepositoryVersionResourceProvider.java
 e440460 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RepositoryEntity.java
 8dc2ec9 
  
ambari-server/src/main/java/org/apache/ambari/server/state/RepositoryInfo.java 
83a 
  
ambari-server/src/main/java/org/apache/ambari/server/state/stack/RepositoryXml.java
 4a0ae3b 
  
ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelper.java
 d26b834 
  ambari-server/src/main/resources/stacks/BIGTOP/0.8/repos/repoinfo.xml 6a385c4 
  
ambari-server/src/main/resources/stacks/HDP/2.0.6.GlusterFS/repos/repoinfo.xml 
9b62568 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/repos/repoinfo.xml 9855074 
  ambari-server/src/main/resources/stacks/HDP/2.0/repos/repoinfo.xml a5d8a51 
  ambari-server/src/main/resources/stacks/HDP/2.1.GlusterFS/repos/repoinfo.xml 
cfee6f5 
  ambari-server/src/main/resources/stacks/HDP/2.1/repos/repoinfo.xml 938f627 
  ambari-server/src/main/resources/stacks/HDP/2.2/repos/repoinfo.xml dbf8506 
  ambari-server/src/main/resources/stacks/HDP/2.3.ECS/repos/repoinfo.xml 
b44cca5 
  ambari-server/src/main/resources/stacks/HDP/2.3.GlusterFS/repos/repoinfo.xml 
24b05b0 
  ambari-server/src/main/resources/stacks/HDP/2.3/repos/repoinfo.xml ec9fba0 
  ambari-server/src/main/resources/stacks/HDP/2.4/repos/repoinfo.xml 1087d9c 
  ambari-server/src/main/resources/stacks/HDP/2.5/repos/repoinfo.xml 8657d91 
  ambari-server/src/main/resources/stacks/HDPWIN/2.1/repos/repoinfo.xml 7f9e075 
  ambari-server/src/main/resources/stacks/HDPWIN/2.2/repos/repoinfo.xml e429adb 
  ambari-server/src/main/resources/stacks/HDPWIN/2.3/repos/repoinfo.xml 54a0bf0 
  ambari-server/src/main/resources/version_definition.xsd bd49028 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/RepositoryVersionResourceProviderTest.java
 8ba141b 
  

Re: Review Request 51761: AMBARI-18050 - Upgrade pre-req check code needs to be decoupled from CheckDescription class

[Di Li]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51761/#review148347
---


Ship it!




Ship It!

- Di Li


On Sept. 9, 2016, 4:59 p.m., Tim Thorpe wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51761/
> ---
> 
> (Updated Sept. 9, 2016, 4:59 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Di Li, Jayush Luniya, and 
> Nate Cole.
> 
> 
> Bugs: AMBARI-18050
> https://issues.apache.org/jira/browse/AMBARI-18050
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Currently each pre-req check must add an entry in the CheckDescription enum. 
> This limits the ability for third party stacks and extensions to provide 
> their own pre-req checks.
> 
> The CheckDescription enum should be rewritten as a class and each pre-req 
> check class should create an instance of it. This will allow stacks and 
> extensions to include their own pre-req checks in separate jar files without 
> requiring changes to ambari-server java code.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/CheckDescription.java
>  aa8e20c 
> 
> Diff: https://reviews.apache.org/r/51761/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test 
> -Dtest=AbstractCheckDescriptorTest,UpgradeCheckOrderTest,CheckHelperTest
> Ran 267 tests in 8.183s
> Total run:1124
> Total errors:0
> Total failures:0
> 
> 
> Thanks,
> 
> Tim Thorpe
> 
>

Review Request 51761: AMBARI-18050 - Upgrade pre-req check code needs to be decoupled from CheckDescription class

[Tim Thorpe]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51761/
---

Review request for Ambari, Alejandro Fernandez, Di Li, Jayush Luniya, and Nate 
Cole.


Bugs: AMBARI-18050
https://issues.apache.org/jira/browse/AMBARI-18050


Repository: ambari


Description
---

Currently each pre-req check must add an entry in the CheckDescription enum. 
This limits the ability for third party stacks and extensions to provide their 
own pre-req checks.

The CheckDescription enum should be rewritten as a class and each pre-req check 
class should create an instance of it. This will allow stacks and extensions to 
include their own pre-req checks in separate jar files without requiring 
changes to ambari-server java code.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/checks/CheckDescription.java
 aa8e20c 

Diff: https://reviews.apache.org/r/51761/diff/


Testing
---

mvn clean test 
-Dtest=AbstractCheckDescriptorTest,UpgradeCheckOrderTest,CheckHelperTest
Ran 267 tests in 8.183s
Total run:1124
Total errors:0
Total failures:0


Thanks,

Tim Thorpe

Re: Review Request 51678: Code review request for AMBARI-18290 (Ambari does not support HBase on HTTPS mode)

[Amarnath reddy pappu]

> On Sept. 7, 2016, 9:05 p.m., Sid Wagle wrote:
> > Ship It!

Thanks for the review. I am closing this request.


- Amarnath reddy


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51678/#review148085
---


On Sept. 7, 2016, 9:02 p.m., Amarnath reddy pappu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51678/
> ---
> 
> (Updated Sept. 7, 2016, 9:02 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, 
> and Sid Wagle.
> 
> 
> Bugs: AMBARI-18290
> https://issues.apache.org/jira/browse/AMBARI-18290
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> If HTTPS is enabled for HBase, Ambari still tries to communicate on HTTP port 
> to fetch JMX properties. because of that Ambari shows incorrect 
> Active/Standby node information.
> 
> Have made code changes to read the "hbase.http.policy" property and then 
> decide the protocol.
> 
> Note: I have not made any changes to Ambari Web. If HTTPS is enabled for 
> HBase then all quick links should be opened on HTTPS. Currently Ambari checks 
>  "dfs.http.policy" HDFS/Hadoop property and decide whether for all the quick 
> links.  it does not check HBase "hbase.http.policy" property to decide. am 
> leaving this functionality as it is.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractProviderModule.java
>  066d5c4 
> 
> Diff: https://reviews.apache.org/r/51678/diff/
> 
> 
> Testing
> ---
> 
> Test cases executed
> --
> 1. Install Ambari2.4, install HBase service, in summary it should show 
> "Active HBase Master"  properly
> 2. Install Ambari2.4, install HBase service and enable HTTPS, in summary it 
> should show "Active HBase Master"  properly
> 3. Install Ambari2.4, install HBase service and add one more HBase master. it 
> should show one node as active and other is standby
> 4. Install Ambari2.4, install HBase service and add one more HBase master and 
> enable HTTPS. it should show one node as active and other is standby
> 
> 
> Thanks,
> 
> Amarnath reddy pappu
> 
>

Re: Review Request 51713: After upgrading cluster from HDP-2.4.x to HDP-2.5.x and added atlas service - missing kafka security properties

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51713/#review148340
---


Ship it!




Ship It!

- Jonathan Hurley


On Sept. 9, 2016, 9:49 a.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51713/
> ---
> 
> (Updated Sept. 9, 2016, 9:49 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, 
> and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-18335
> https://issues.apache.org/jira/browse/AMBARI-18335
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Steps to repro:
> - Install Ambari 2.2.2
> - Install HDP-2.4.x cluster with Atlas
> - Stop Atlas
> - Upgrade Ambari to 2.4
> - Delete Atlas service
> - Upgrade the cluster to HDP-2.5.x cluster
> - Add Atlas service.
> 
> *Below config properties are missing from atlas-applicataion.properties file 
> for Atlas, Storm, Falcon, Hive services.*
> ```
> atlas.jaas.KafkaClient.option.keyTab = 
> /etc/security/keytabs/atlas.service.keytab
> atlas.jaas.KafkaClient.option.principal = atlas/_h...@example.com
> ```
> 
> From HDP 2.4 to 2.5, the kerberos.json file for Atlas changed.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/UpgradeUserKerberosDescriptor.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptor.java
>  84a9111 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptorContainer.java
>  39ebdaf 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptor.java
>  3cdd9908 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptor.java
>  484f65c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelper.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptor.java
>  7ce1c9f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptor.java
>  0156e4a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosServiceDescriptor.java
>  72dbcfe 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/kerberos.json
>  d569447 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml
>  ad9fc97 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> 27f2010 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml
>  4df5fef 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> b8c51f5 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HBASE/kerberos.json 
> 501bcd3 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.5.xml
>  0b6f762 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.5.xml 
> 49e9d87 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterKerberosDescriptorResourceProviderTest.java
>  f551b42 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosComponentDescriptorTest.java
>  b74f417 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptorTest.java
>  6017fae 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorTest.java
>  9463749 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelperTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosIdentityDescriptorTest.java
>  874da31 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptorTest.java
>  c10d106 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptorTest.java
>  5c249e2 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosServiceDescriptorTest.java
>  e1af515 
> 
> Diff: https://reviews.apache.org/r/51713/diff/
> 
> 
> Testing
> ---
> 
> unit tests, manually upgrades
> 
> # Local test results: 
> 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 1:20:19.796s
> [INFO] Finished at: Fri Sep 09 09:29:44 EDT 2016
> [INFO] Final Memory: 62M/1882M
> [INFO] 
> 
> 
> # 

Re: Review Request 51713: After upgrading cluster from HDP-2.4.x to HDP-2.5.x and added atlas service - missing kafka security properties

[Nate Cole]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51713/#review148339
---


Fix it, then Ship it!





ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/UpgradeUserKerberosDescriptor.java
 (line 134)


Just add a bit more information to some of these messages, if possible:

"The Kerberos descriptor for stack %s was not found."

etc.  These show as stdout/stderr for the task, so as much detail that we 
can get there, the easier it is to pinpoint what happened.


- Nate Cole


On Sept. 9, 2016, 9:49 a.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51713/
> ---
> 
> (Updated Sept. 9, 2016, 9:49 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, 
> and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-18335
> https://issues.apache.org/jira/browse/AMBARI-18335
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Steps to repro:
> - Install Ambari 2.2.2
> - Install HDP-2.4.x cluster with Atlas
> - Stop Atlas
> - Upgrade Ambari to 2.4
> - Delete Atlas service
> - Upgrade the cluster to HDP-2.5.x cluster
> - Add Atlas service.
> 
> *Below config properties are missing from atlas-applicataion.properties file 
> for Atlas, Storm, Falcon, Hive services.*
> ```
> atlas.jaas.KafkaClient.option.keyTab = 
> /etc/security/keytabs/atlas.service.keytab
> atlas.jaas.KafkaClient.option.principal = atlas/_h...@example.com
> ```
> 
> From HDP 2.4 to 2.5, the kerberos.json file for Atlas changed.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/UpgradeUserKerberosDescriptor.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptor.java
>  84a9111 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptorContainer.java
>  39ebdaf 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptor.java
>  3cdd9908 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptor.java
>  484f65c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelper.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptor.java
>  7ce1c9f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptor.java
>  0156e4a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosServiceDescriptor.java
>  72dbcfe 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/kerberos.json
>  d569447 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml
>  ad9fc97 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> 27f2010 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml
>  4df5fef 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> b8c51f5 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HBASE/kerberos.json 
> 501bcd3 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.5.xml
>  0b6f762 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.5.xml 
> 49e9d87 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterKerberosDescriptorResourceProviderTest.java
>  f551b42 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosComponentDescriptorTest.java
>  b74f417 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptorTest.java
>  6017fae 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorTest.java
>  9463749 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelperTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosIdentityDescriptorTest.java
>  874da31 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptorTest.java
>  c10d106 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptorTest.java
>  5c249e2 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosServiceDescriptorTest.java
>  e1af515 
> 
> Diff: https://reviews.apache.org/r/51713/diff/
> 
> 
> Testing
> ---
> 
> unit 

Re: Review Request 51748: Ranger audit log collection is not used in Log Search (managed by ambari)

[Robert Nettleton]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51748/#review148337
---


Ship it!




Ship It!

- Robert Nettleton


On Sept. 8, 2016, 10:02 p.m., Miklos Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51748/
> ---
> 
> (Updated Sept. 8, 2016, 10:02 p.m.)
> 
> 
> Review request for Ambari, Oliver Szabo, Robert Nettleton, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-18348
> https://issues.apache.org/jira/browse/AMBARI-18348
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Audit log collection and ranger audit log collection are 2 different 
> collection with almost the same schema. With solr alias support, we can use 
> both.
> Also use ambari audit logs inside audit collection, not in service log 
> collection
> 
> 
> Diffs
> -
> 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/filter/FilterKeyValue.java
>  c9c3f2c 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/test/java/org/apache/ambari/logfeeder/filter/FilterKeyValueTest.java
>  30cee42 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-properties.xml
>  e521c8e 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/params.py
>  8b0ce75 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch.py
>  8d84093 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/templates/input.config-ambari.json.j2
>  34f4b30 
>   ambari-server/src/test/python/stacks/2.4/configs/default.json 8ac6fa7 
>   ambari-web/app/data/HDP2/site_properties.js 9311e75 
> 
> Diff: https://reviews.apache.org/r/51748/diff/
> 
> 
> Testing
> ---
> 
> Tested on local cluster.
> 
> 
> Thanks,
> 
> Miklos Gergely
> 
>

Re: Review Request 51713: After upgrading cluster from HDP-2.4.x to HDP-2.5.x and added atlas service - missing kafka security properties

[Robert Levas]

> On Sept. 8, 2016, 9:07 p.m., Sumit Mohanty wrote:
> > Impressive set of changes. One question:
> > Some of the config changes for secure cluster are fed from the stack 
> > advisor. With this capability, new artifacts in kerberos.json file will be 
> > added during HDP upgrade. I assume there is no stack advisor call to go 
> > with it. Will that be an issue?

There are no stack advisor updates needed related to this. Any updates to the 
stack advisor related to HDP 2.5 would have been added in a different patch. 
This issue is only related to the user-specified Kerberos descritor in how it 
becomes out of sync with the stack Kerberos descriptor once the stack version 
changes.


- Robert


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51713/#review148295
---


On Sept. 9, 2016, 9:42 a.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51713/
> ---
> 
> (Updated Sept. 9, 2016, 9:42 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, 
> and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-18335
> https://issues.apache.org/jira/browse/AMBARI-18335
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Steps to repro:
> - Install Ambari 2.2.2
> - Install HDP-2.4.x cluster with Atlas
> - Stop Atlas
> - Upgrade Ambari to 2.4
> - Delete Atlas service
> - Upgrade the cluster to HDP-2.5.x cluster
> - Add Atlas service.
> 
> *Below config properties are missing from atlas-applicataion.properties file 
> for Atlas, Storm, Falcon, Hive services.*
> ```
> atlas.jaas.KafkaClient.option.keyTab = 
> /etc/security/keytabs/atlas.service.keytab
> atlas.jaas.KafkaClient.option.principal = atlas/_h...@example.com
> ```
> 
> From HDP 2.4 to 2.5, the kerberos.json file for Atlas changed.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/UpgradeUserKerberosDescriptor.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptor.java
>  84a9111 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptorContainer.java
>  39ebdaf 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptor.java
>  3cdd9908 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptor.java
>  484f65c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelper.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptor.java
>  7ce1c9f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptor.java
>  0156e4a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosServiceDescriptor.java
>  72dbcfe 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/kerberos.json
>  d569447 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml
>  ad9fc97 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> 27f2010 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml
>  4df5fef 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> b8c51f5 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HBASE/kerberos.json 
> 501bcd3 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.5.xml
>  0b6f762 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.5.xml 
> 49e9d87 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterKerberosDescriptorResourceProviderTest.java
>  f551b42 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosComponentDescriptorTest.java
>  b74f417 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptorTest.java
>  6017fae 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorTest.java
>  9463749 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelperTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosIdentityDescriptorTest.java
>  874da31 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptorTest.java
>  c10d106 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptorTest.java
>  5c249e2 
>   
> 

Re: Review Request 51713: After upgrading cluster from HDP-2.4.x to HDP-2.5.x and added atlas service - missing kafka security properties

[Robert Levas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51713/
---

(Updated Sept. 9, 2016, 9:49 a.m.)


Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, and 
Sumit Mohanty.


Bugs: AMBARI-18335
https://issues.apache.org/jira/browse/AMBARI-18335


Repository: ambari


Description
---

Steps to repro:
- Install Ambari 2.2.2
- Install HDP-2.4.x cluster with Atlas
- Stop Atlas
- Upgrade Ambari to 2.4
- Delete Atlas service
- Upgrade the cluster to HDP-2.5.x cluster
- Add Atlas service.

*Below config properties are missing from atlas-applicataion.properties file 
for Atlas, Storm, Falcon, Hive services.*
```
atlas.jaas.KafkaClient.option.keyTab = 
/etc/security/keytabs/atlas.service.keytab
atlas.jaas.KafkaClient.option.principal = atlas/_h...@example.com
```

>From HDP 2.4 to 2.5, the kerberos.json file for Atlas changed.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/UpgradeUserKerberosDescriptor.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptor.java
 84a9111 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptorContainer.java
 39ebdaf 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptor.java
 3cdd9908 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptor.java
 484f65c 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelper.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptor.java
 7ce1c9f 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptor.java
 0156e4a 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosServiceDescriptor.java
 72dbcfe 
  
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/kerberos.json 
d569447 
  
ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml
 ad9fc97 
  ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
27f2010 
  
ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml
 4df5fef 
  ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
b8c51f5 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/HBASE/kerberos.json 
501bcd3 
  
ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.5.xml
 0b6f762 
  ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.5.xml 
49e9d87 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterKerberosDescriptorResourceProviderTest.java
 f551b42 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosComponentDescriptorTest.java
 b74f417 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptorTest.java
 6017fae 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorTest.java
 9463749 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelperTest.java
 PRE-CREATION 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosIdentityDescriptorTest.java
 874da31 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptorTest.java
 c10d106 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptorTest.java
 5c249e2 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosServiceDescriptorTest.java
 e1af515 

Diff: https://reviews.apache.org/r/51713/diff/


Testing (updated)
---

unit tests, manually upgrades

# Local test results: 

[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 1:20:19.796s
[INFO] Finished at: Fri Sep 09 09:29:44 EDT 2016
[INFO] Final Memory: 62M/1882M
[INFO] 

# Jenkins test results: PENDING


Thanks,

Robert Levas

Re: Review Request 51713: After upgrading cluster from HDP-2.4.x to HDP-2.5.x and added atlas service - missing kafka security properties

[Robert Levas]

> On Sept. 8, 2016, 12:06 p.m., Nate Cole wrote:
> > What will happen if, say, doing an Upgrade after this code has run, the 
> > user decides to Downgrade?  Will the old data be intact or does it get 
> > overwritten?
> 
> Robert Levas wrote:
> Looking at the *-upgrade-2.5.xml files, will 
> `UpgradeUserKerberosDescriptor` execute for downgrades? I need to test that. 
> However, theorectially, the Kerberos descriptor will be updated to work with 
> the target stack.  So it should be fine.

I tested this scneario with changes in rev3.  It works as expected in both 
directions now.


- Robert


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51713/#review148199
---


On Sept. 9, 2016, 9:42 a.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51713/
> ---
> 
> (Updated Sept. 9, 2016, 9:42 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, 
> and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-18335
> https://issues.apache.org/jira/browse/AMBARI-18335
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Steps to repro:
> - Install Ambari 2.2.2
> - Install HDP-2.4.x cluster with Atlas
> - Stop Atlas
> - Upgrade Ambari to 2.4
> - Delete Atlas service
> - Upgrade the cluster to HDP-2.5.x cluster
> - Add Atlas service.
> 
> *Below config properties are missing from atlas-applicataion.properties file 
> for Atlas, Storm, Falcon, Hive services.*
> ```
> atlas.jaas.KafkaClient.option.keyTab = 
> /etc/security/keytabs/atlas.service.keytab
> atlas.jaas.KafkaClient.option.principal = atlas/_h...@example.com
> ```
> 
> From HDP 2.4 to 2.5, the kerberos.json file for Atlas changed.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/UpgradeUserKerberosDescriptor.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptor.java
>  84a9111 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptorContainer.java
>  39ebdaf 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptor.java
>  3cdd9908 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptor.java
>  484f65c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelper.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptor.java
>  7ce1c9f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptor.java
>  0156e4a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosServiceDescriptor.java
>  72dbcfe 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/kerberos.json
>  d569447 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml
>  ad9fc97 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> 27f2010 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml
>  4df5fef 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> b8c51f5 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HBASE/kerberos.json 
> 501bcd3 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.5.xml
>  0b6f762 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.5.xml 
> 49e9d87 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterKerberosDescriptorResourceProviderTest.java
>  f551b42 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosComponentDescriptorTest.java
>  b74f417 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptorTest.java
>  6017fae 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorTest.java
>  9463749 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelperTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosIdentityDescriptorTest.java
>  874da31 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptorTest.java
>  c10d106 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptorTest.java
>  5c249e2 
>   
> 

Re: Review Request 51713: After upgrading cluster from HDP-2.4.x to HDP-2.5.x and added atlas service - missing kafka security properties

[Robert Levas]

> On Sept. 8, 2016, 1:07 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelper.java,
> >  lines 555-558
> > 
> >
> > I'm worried about this logic on downgrade. When upgrading from 2.3 to 
> > 2.5, the previous value is from 2.3 and the new value is from 2.5.
> > 
> > However a downgrade also has the same values (from=2.3, to=2.5) but the 
> > direction changes to DOWNGRADE.
> > 
> > Doesn't that mean that this needs to check direction as well?
> 
> Robert Levas wrote:
> On a downgrade (kicked off during the middle of the upgrade process), the 
> previous and new versions appear to both the original version.  Is there a 
> way to know the versions explicity rather than asking the cluster object?
> 
> Jonathan Hurley wrote:
> I don't think so - you'd need to pass in information from the upgrade 
> context into the Kerberos helper and query that object for the stacks and 
> direction.

Thanks to Nate (@ncole), I found the context properties that explicitly 
indicates target and source versions and the upgrade direction.


- Robert


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51713/#review148193
---


On Sept. 9, 2016, 9:42 a.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51713/
> ---
> 
> (Updated Sept. 9, 2016, 9:42 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, 
> and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-18335
> https://issues.apache.org/jira/browse/AMBARI-18335
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Steps to repro:
> - Install Ambari 2.2.2
> - Install HDP-2.4.x cluster with Atlas
> - Stop Atlas
> - Upgrade Ambari to 2.4
> - Delete Atlas service
> - Upgrade the cluster to HDP-2.5.x cluster
> - Add Atlas service.
> 
> *Below config properties are missing from atlas-applicataion.properties file 
> for Atlas, Storm, Falcon, Hive services.*
> ```
> atlas.jaas.KafkaClient.option.keyTab = 
> /etc/security/keytabs/atlas.service.keytab
> atlas.jaas.KafkaClient.option.principal = atlas/_h...@example.com
> ```
> 
> From HDP 2.4 to 2.5, the kerberos.json file for Atlas changed.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/UpgradeUserKerberosDescriptor.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptor.java
>  84a9111 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptorContainer.java
>  39ebdaf 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptor.java
>  3cdd9908 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptor.java
>  484f65c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelper.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptor.java
>  7ce1c9f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptor.java
>  0156e4a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosServiceDescriptor.java
>  72dbcfe 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/kerberos.json
>  d569447 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml
>  ad9fc97 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> 27f2010 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml
>  4df5fef 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> b8c51f5 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HBASE/kerberos.json 
> 501bcd3 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.5.xml
>  0b6f762 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.5.xml 
> 49e9d87 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterKerberosDescriptorResourceProviderTest.java
>  f551b42 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosComponentDescriptorTest.java
>  b74f417 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptorTest.java
>  6017fae 
>   
> 

Re: Review Request 51713: After upgrading cluster from HDP-2.4.x to HDP-2.5.x and added atlas service - missing kafka security properties

[Robert Levas]

> On Sept. 8, 2016, 11:13 a.m., Nate Cole wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml,
> >  line 472
> > 
> >
> > Can remove the "(if needed)" in text.

done.


- Robert


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51713/#review148190
---


On Sept. 9, 2016, 9:42 a.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51713/
> ---
> 
> (Updated Sept. 9, 2016, 9:42 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, 
> and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-18335
> https://issues.apache.org/jira/browse/AMBARI-18335
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Steps to repro:
> - Install Ambari 2.2.2
> - Install HDP-2.4.x cluster with Atlas
> - Stop Atlas
> - Upgrade Ambari to 2.4
> - Delete Atlas service
> - Upgrade the cluster to HDP-2.5.x cluster
> - Add Atlas service.
> 
> *Below config properties are missing from atlas-applicataion.properties file 
> for Atlas, Storm, Falcon, Hive services.*
> ```
> atlas.jaas.KafkaClient.option.keyTab = 
> /etc/security/keytabs/atlas.service.keytab
> atlas.jaas.KafkaClient.option.principal = atlas/_h...@example.com
> ```
> 
> From HDP 2.4 to 2.5, the kerberos.json file for Atlas changed.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/UpgradeUserKerberosDescriptor.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptor.java
>  84a9111 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptorContainer.java
>  39ebdaf 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptor.java
>  3cdd9908 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptor.java
>  484f65c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelper.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptor.java
>  7ce1c9f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptor.java
>  0156e4a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosServiceDescriptor.java
>  72dbcfe 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/kerberos.json
>  d569447 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml
>  ad9fc97 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> 27f2010 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml
>  4df5fef 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> b8c51f5 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HBASE/kerberos.json 
> 501bcd3 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.5.xml
>  0b6f762 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.5.xml 
> 49e9d87 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterKerberosDescriptorResourceProviderTest.java
>  f551b42 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosComponentDescriptorTest.java
>  b74f417 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptorTest.java
>  6017fae 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorTest.java
>  9463749 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelperTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosIdentityDescriptorTest.java
>  874da31 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptorTest.java
>  c10d106 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptorTest.java
>  5c249e2 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosServiceDescriptorTest.java
>  e1af515 
> 
> Diff: https://reviews.apache.org/r/51713/diff/
> 
> 
> Testing
> ---
> 
> unit tests, manually upgrades
> 
> # Local test results: 
> 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 

Re: Review Request 51725: Microsoft-R client should work in a secured cluster

[Attila Doroszlai]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51725/#review148329
---




contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/package/scripts/params_linux.py
 (line 49)


Is this necessary?



contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/package/scripts/service_check.py
 (line 32)


Seems unnecessary, `HdfsResource` is used from `params`



contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/package/scripts/service_check.py
 (line 45)


* Indent by 2 spaces, not 4
 * Would be nice to extract `/user/RevoShare` to a variable in `params`


- Attila Doroszlai


On Sept. 9, 2016, 8:16 a.m., Balázs Bence Sári wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51725/
> ---
> 
> (Updated Sept. 9, 2016, 8:16 a.m.)
> 
> 
> Review request for Ambari, Attila Doroszlai, Jayush Luniya, Oliver Szabo, 
> Sumit Mohanty, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-18338
> https://issues.apache.org/jira/browse/AMBARI-18338
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> - Added kerberos json to make sure the client can use the smokeuser and hdfs 
> keytabs. 
> - Added the creation of /user/RevoShare hdfs directory to the service check 
> (with hdfs user/principal)
> - Extended stack support to HDP 2.4 and 2.5
> 
> 
> Diffs
> -
> 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/kerberos.json
>  PRE-CREATION 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/package/scripts/params_linux.py
>  d72c42e 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/package/scripts/service_check.py
>  8a425ca 
> 
> Diff: https://reviews.apache.org/r/51725/diff/
> 
> 
> Testing
> ---
> 
> Manually tested the installation and kerberization of MSFT-R via UI and 
> blueprints on 1-8 node clusters.
> 
> 
> Thanks,
> 
> Balázs Bence Sári
> 
>

Re: Review Request 51713: After upgrading cluster from HDP-2.4.x to HDP-2.5.x and added atlas service - missing kafka security properties

[Robert Levas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51713/
---

(Updated Sept. 9, 2016, 9:42 a.m.)


Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, and 
Sumit Mohanty.


Changes
---

changes made due to reviewer comments

- fixed logger's associated class name in `UpgradeUserKerberosDescriptor`
- fixed messages logged in `UpgradeUserKerberosDescriptor`
- added ability to properly handle downgrades


Bugs: AMBARI-18335
https://issues.apache.org/jira/browse/AMBARI-18335


Repository: ambari


Description
---

Steps to repro:
- Install Ambari 2.2.2
- Install HDP-2.4.x cluster with Atlas
- Stop Atlas
- Upgrade Ambari to 2.4
- Delete Atlas service
- Upgrade the cluster to HDP-2.5.x cluster
- Add Atlas service.

*Below config properties are missing from atlas-applicataion.properties file 
for Atlas, Storm, Falcon, Hive services.*
```
atlas.jaas.KafkaClient.option.keyTab = 
/etc/security/keytabs/atlas.service.keytab
atlas.jaas.KafkaClient.option.principal = atlas/_h...@example.com
```

>From HDP 2.4 to 2.5, the kerberos.json file for Atlas changed.


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/UpgradeUserKerberosDescriptor.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptor.java
 84a9111 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptorContainer.java
 39ebdaf 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptor.java
 3cdd9908 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptor.java
 484f65c 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelper.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptor.java
 7ce1c9f 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptor.java
 0156e4a 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosServiceDescriptor.java
 72dbcfe 
  
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/kerberos.json 
d569447 
  
ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml
 ad9fc97 
  ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
27f2010 
  
ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml
 4df5fef 
  ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
b8c51f5 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/HBASE/kerberos.json 
501bcd3 
  
ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.5.xml
 0b6f762 
  ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.5.xml 
49e9d87 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterKerberosDescriptorResourceProviderTest.java
 f551b42 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosComponentDescriptorTest.java
 b74f417 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosConfigurationDescriptorTest.java
 6017fae 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorTest.java
 9463749 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosDescriptorUpdateHelperTest.java
 PRE-CREATION 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosIdentityDescriptorTest.java
 874da31 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptorTest.java
 c10d106 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptorTest.java
 5c249e2 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/KerberosServiceDescriptorTest.java
 e1af515 

Diff: https://reviews.apache.org/r/51713/diff/


Testing
---

unit tests, manually upgrades

# Local test results: 

[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 1:22:13.782s
[INFO] Finished at: Wed Sep 07 22:51:30 EDT 2016
[INFO] Final Memory: 60M/1835M
[INFO] 

# Jenkins test results: PENDING


Thanks,

Robert Levas

Re: Review Request 51745: Setting fetch_nonlocal_groups to false Can Prevent Services From Starting

[Andrew Onischuk]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51745/#review148320
---


Ship it!




Ship It!

- Andrew Onischuk


On Sept. 8, 2016, 8:46 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51745/
> ---
> 
> (Updated Sept. 8, 2016, 8:46 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Dmitro Lisnichenko, and Nate Cole.
> 
> 
> Bugs: AMBARI-18347
> https://issues.apache.org/jira/browse/AMBARI-18347
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When setting the {{cluster-env/fetch_nonlocal_groups}} value to {{false}}, if 
> the {{/etc/group}} file has an unexpected format, services will fail to start:
> 
> ```
> File "/usr/lib/python2.6/site-packages/resource_management/core/utils.py", 
> line 146, in decorated 
> v = undecorated(self) 
> File 
> "/usr/lib/python2.6/site-packages/resource_management/core/providers/accounts.py",
>  line 109, in user_groups 
> group_users = entries[3].split(',') 
> ```
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/test/python/resource_management/TestUserResource.py 
> da94521 
>   
> ambari-common/src/main/python/resource_management/core/providers/accounts.py 
> 5169b12 
> 
> Diff: https://reviews.apache.org/r/51745/diff/
> 
> 
> Testing
> ---
> 
> --
> Total run:1124
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Re: Review Request 51745: Setting fetch_nonlocal_groups to false Can Prevent Services From Starting

[Andrew Onischuk]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51745/#review148319
---




ambari-common/src/main/python/resource_management/core/providers/accounts.py 
(line 112)


"in case the line is valid" I think that what you meant to say.


- Andrew Onischuk


On Sept. 8, 2016, 8:46 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51745/
> ---
> 
> (Updated Sept. 8, 2016, 8:46 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Dmitro Lisnichenko, and Nate Cole.
> 
> 
> Bugs: AMBARI-18347
> https://issues.apache.org/jira/browse/AMBARI-18347
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When setting the {{cluster-env/fetch_nonlocal_groups}} value to {{false}}, if 
> the {{/etc/group}} file has an unexpected format, services will fail to start:
> 
> ```
> File "/usr/lib/python2.6/site-packages/resource_management/core/utils.py", 
> line 146, in decorated 
> v = undecorated(self) 
> File 
> "/usr/lib/python2.6/site-packages/resource_management/core/providers/accounts.py",
>  line 109, in user_groups 
> group_users = entries[3].split(',') 
> ```
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/test/python/resource_management/TestUserResource.py 
> da94521 
>   
> ambari-common/src/main/python/resource_management/core/providers/accounts.py 
> 5169b12 
> 
> Diff: https://reviews.apache.org/r/51745/diff/
> 
> 
> Testing
> ---
> 
> --
> Total run:1124
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Re: Review Request 51725: Microsoft-R client should work in a secured cluster

[Sebastian Toader]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51725/#review148317
---


Ship it!




Ship It!

- Sebastian Toader


On Sept. 9, 2016, 10:16 a.m., Balázs Bence Sári wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51725/
> ---
> 
> (Updated Sept. 9, 2016, 10:16 a.m.)
> 
> 
> Review request for Ambari, Attila Doroszlai, Jayush Luniya, Oliver Szabo, 
> Sumit Mohanty, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-18338
> https://issues.apache.org/jira/browse/AMBARI-18338
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> - Added kerberos json to make sure the client can use the smokeuser and hdfs 
> keytabs. 
> - Added the creation of /user/RevoShare hdfs directory to the service check 
> (with hdfs user/principal)
> - Extended stack support to HDP 2.4 and 2.5
> 
> 
> Diffs
> -
> 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/kerberos.json
>  PRE-CREATION 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/package/scripts/params_linux.py
>  d72c42e 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/package/scripts/service_check.py
>  8a425ca 
> 
> Diff: https://reviews.apache.org/r/51725/diff/
> 
> 
> Testing
> ---
> 
> Manually tested the installation and kerberization of MSFT-R via UI and 
> blueprints on 1-8 node clusters.
> 
> 
> Thanks,
> 
> Balázs Bence Sári
> 
>

Re: Review Request 51745: Setting fetch_nonlocal_groups to false Can Prevent Services From Starting

[Dmitro Lisnichenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51745/#review148315
---


Ship it!




Ship It!

- Dmitro Lisnichenko


On Sept. 8, 2016, 11:46 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51745/
> ---
> 
> (Updated Sept. 8, 2016, 11:46 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Dmitro Lisnichenko, and Nate Cole.
> 
> 
> Bugs: AMBARI-18347
> https://issues.apache.org/jira/browse/AMBARI-18347
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When setting the {{cluster-env/fetch_nonlocal_groups}} value to {{false}}, if 
> the {{/etc/group}} file has an unexpected format, services will fail to start:
> 
> ```
> File "/usr/lib/python2.6/site-packages/resource_management/core/utils.py", 
> line 146, in decorated 
> v = undecorated(self) 
> File 
> "/usr/lib/python2.6/site-packages/resource_management/core/providers/accounts.py",
>  line 109, in user_groups 
> group_users = entries[3].split(',') 
> ```
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/test/python/resource_management/TestUserResource.py 
> da94521 
>   
> ambari-common/src/main/python/resource_management/core/providers/accounts.py 
> 5169b12 
> 
> Diff: https://reviews.apache.org/r/51745/diff/
> 
> 
> Testing
> ---
> 
> --
> Total run:1124
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Re: Review Request 51725: Microsoft-R client should work in a secured cluster

[Balázs Bence Sári]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51725/
---

(Updated Sept. 9, 2016, 8:16 a.m.)


Review request for Ambari, Attila Doroszlai, Jayush Luniya, Oliver Szabo, Sumit 
Mohanty, and Sebastian Toader.


Bugs: AMBARI-18338
https://issues.apache.org/jira/browse/AMBARI-18338


Repository: ambari


Description
---

- Added kerberos json to make sure the client can use the smokeuser and hdfs 
keytabs. 
- Added the creation of /user/RevoShare hdfs directory to the service check 
(with hdfs user/principal)
- Extended stack support to HDP 2.4 and 2.5


Diffs (updated)
-

  
contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/kerberos.json
 PRE-CREATION 
  
contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/package/scripts/params_linux.py
 d72c42e 
  
contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/package/scripts/service_check.py
 8a425ca 

Diff: https://reviews.apache.org/r/51725/diff/


Testing
---

Manually tested the installation and kerberization of MSFT-R via UI and 
blueprints on 1-8 node clusters.


Thanks,

Balázs Bence Sári

Re: Review Request 51725: Microsoft-R client should work in a secured cluster

[Balázs Bence Sári]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51725/
---

(Updated Sept. 9, 2016, 8:16 a.m.)


Review request for Ambari, Attila Doroszlai, Jayush Luniya, Oliver Szabo, Sumit 
Mohanty, and Sebastian Toader.


Changes
---

Fixed a review finding.


Bugs: AMBARI-18338
https://issues.apache.org/jira/browse/AMBARI-18338


Repository: ambari


Description
---

- Added kerberos json to make sure the client can use the smokeuser and hdfs 
keytabs. 
- Added the creation of /user/RevoShare hdfs directory to the service check 
(with hdfs user/principal)
- Extended stack support to HDP 2.4 and 2.5


Diffs (updated)
-

  
contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/kerberos.json
 PRE-CREATION 
  
contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/package/scripts/params_linux.py
 d72c42e 
  
contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/package/scripts/service_check.py
 8a425ca 

Diff: https://reviews.apache.org/r/51725/diff/


Testing
---

Manually tested the installation and kerberization of MSFT-R via UI and 
blueprints on 1-8 node clusters.


Thanks,

Balázs Bence Sári