Review Request 60937: Add Exporting Hosts when Registering Host and Adding Host

[Xiang Ji]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60937/
---

Review request for Ambari, Alexandr Antonenko, Andrii Tkach, Jaimin Jetly, 
Richard Zang, and Yusaku Sako.


Bugs: AMBARI-21505
https://issues.apache.org/jira/browse/AMBARI-21505


Repository: ambari


Description
---

On a big cluster (e.g. on a 1000-node cluster), it's necessary to filter hosts 
based on status when registering host and adding host. So add one button to 
export running-hosts, registering-hosts, registered-hosts, failed-hosts and 
all-hosts.


Diffs
-

  ambari-web/app/controllers/wizard/step3_controller.js 29393e3 
  ambari-web/app/messages.js f75d801 
  ambari-web/app/templates/wizard/step3.hbs e00012c 
  ambari-web/app/views/wizard/step3_view.js 3249b24 


Diff: https://reviews.apache.org/r/60937/diff/1/


Testing
---

Done


Thanks,

Xiang Ji

Re: Review Request 60919: AMBARI-21502. Cross-stack migration from BigInsights to HDP, EU needs to set hive-site custom.hive.warehouse.mode to 0770

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60919/#review180785
---


Ship it!




Ship It!

- Jonathan Hurley


On July 17, 2017, 5:31 p.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60919/
> ---
> 
> (Updated July 17, 2017, 5:31 p.m.)
> 
> 
> Review request for Ambari, Andrii Tkach, Dmytro Grinenko, Jonathan Hurley, 
> Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-21502
> https://issues.apache.org/jira/browse/AMBARI-21502
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> During cross stack upgrade from BigInsights, if user has not set 
> custom.hive.warehouse.mode in hive-site, as the default value is changed in 
> HDFS from BigInsights default (0770) to HDP default (0777).
> EU should set the value of custom.hive.warehouse.mode to 0770
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/config-upgrade.xml
>  f1a954e 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
>  5f1e06c 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/config-upgrade.xml
>  310e504 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
>  5b8f8d9 
> 
> 
> Diff: https://reviews.apache.org/r/60919/diff/2/
> 
> 
> Testing
> ---
> 
> Python unit tests passed.
> 
> --
> Total run:1161
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Re: Review Request 60921: Restart of MR2 History Server failed due to null in immutable_paths

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60921/#review180757
---




ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java
Lines 228 (patched)


Should we add a log here that some property is being removed as its 
empty/null.



ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py
Lines 100 (patched)


With the java change, will this error path be hit?


- Sumit Mohanty


On July 17, 2017, 8:14 p.m., Vitalyi Brodetskyi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60921/
> ---
> 
> (Updated July 17, 2017, 8:14 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-21504
> https://issues.apache.org/jira/browse/AMBARI-21504
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Steps:
> 
> * Installed Ambari 2.2 with Slider and services it required
> * Upgraded Ambari to 2.5.2.0-134
> * Registered HDP 2.6.1.0 repo, installed packages
> * Restarted services that needed restart
> * Ran service checks
> * Started upgrade
> 
> Result: _Restarting History Server_ step failed with 
> 
> {noformat:title=errors-91.txt}
> Traceback (most recent call last):
>   File 
> "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py",
>  line 134, in 
> HistoryServer().execute()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 329, in execute
> method(env)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 841, in restart
> self.pre_upgrade_restart(env, upgrade_type=upgrade_type)
>   File 
> "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py",
>  line 85, in pre_upgrade_restart
> copy_to_hdfs("mapreduce", params.user_group, params.hdfs_user, 
> skip=params.sysprep_skip_copy_tarballs_hdfs)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/copy_tarball.py",
>  line 256, in copy_to_hdfs
> mode=0555
>   File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", 
> line 155, in __init__
> self.env.run()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
> line 160, in run
> self.run_action(resource, action)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
> line 124, in run_action
> provider_action()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py",
>  line 560, in action_create_on_execute
> self.action_delayed("create")
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py",
>  line 550, in action_delayed
> parsed_not_managed_paths = [HdfsResourceProvider.parse_path(path) for 
> path in self.resource.immutable_paths]
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py",
>  line 519, in parse_path
> math_with_protocol_and_nn_url = re.match("[a-zA-Z]+://[^/]+(/.+)", path)
>   File "/usr/lib64/python2.7/re.py", line 137, in match
> return _compile(pattern, flags).match(string)
> TypeError: expected string or buffer
> {noformat}
> 
> {noformat:title=command-91.json}
> "hostLevelParams": {
> ...
> "not_managed_hdfs_path_list": "[null,\"/mr-history/done\",\"/app-logs\"]",
> ...
> }
> {noformat}
> 
> Failure seems to be due to {{null}} value in the list.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java
>  9b19ade 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  82171f6 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClientConfigResourceProvider.java
>  daca2d0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 
> ab8026c 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py
>  9a9f9ca 
>   
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/service.py
>  6495209 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/ConfigHelperTest.java
>  b1c10f5 
> 
> 
> Diff: https://reviews.apache.org/r/60921/diff/1/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>

Re: Review Request 60919: AMBARI-21502. Cross-stack migration from BigInsights to HDP, EU needs to set hive-site custom.hive.warehouse.mode to 0770

[Dmytro Grinenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60919/#review180748
---


Ship it!




Ship It!

- Dmytro Grinenko


On July 17, 2017, 9:31 p.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60919/
> ---
> 
> (Updated July 17, 2017, 9:31 p.m.)
> 
> 
> Review request for Ambari, Andrii Tkach, Dmytro Grinenko, Jonathan Hurley, 
> Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-21502
> https://issues.apache.org/jira/browse/AMBARI-21502
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> During cross stack upgrade from BigInsights, if user has not set 
> custom.hive.warehouse.mode in hive-site, as the default value is changed in 
> HDFS from BigInsights default (0770) to HDP default (0777).
> EU should set the value of custom.hive.warehouse.mode to 0770
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/config-upgrade.xml
>  f1a954e 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
>  5f1e06c 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/config-upgrade.xml
>  310e504 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
>  5b8f8d9 
> 
> 
> Diff: https://reviews.apache.org/r/60919/diff/2/
> 
> 
> Testing
> ---
> 
> Python unit tests passed.
> 
> --
> Total run:1161
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Re: Review Request 60919: AMBARI-21502. Cross-stack migration from BigInsights to HDP, EU needs to set hive-site custom.hive.warehouse.mode to 0770

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60919/#review180747
---


Ship it!




Ship It!

- Sumit Mohanty


On July 17, 2017, 9:31 p.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60919/
> ---
> 
> (Updated July 17, 2017, 9:31 p.m.)
> 
> 
> Review request for Ambari, Andrii Tkach, Dmytro Grinenko, Jonathan Hurley, 
> Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-21502
> https://issues.apache.org/jira/browse/AMBARI-21502
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> During cross stack upgrade from BigInsights, if user has not set 
> custom.hive.warehouse.mode in hive-site, as the default value is changed in 
> HDFS from BigInsights default (0770) to HDP default (0777).
> EU should set the value of custom.hive.warehouse.mode to 0770
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/config-upgrade.xml
>  f1a954e 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
>  5f1e06c 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/config-upgrade.xml
>  310e504 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
>  5b8f8d9 
> 
> 
> Diff: https://reviews.apache.org/r/60919/diff/2/
> 
> 
> Testing
> ---
> 
> Python unit tests passed.
> 
> --
> Total run:1161
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Re: Review Request 60919: AMBARI-21502. Cross-stack migration from BigInsights to HDP, EU needs to set hive-site custom.hive.warehouse.mode to 0770

[Alejandro Fernandez]

> On July 17, 2017, 8:16 p.m., Sumit Mohanty wrote:
> > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/config-upgrade.xml
> > Lines 177 (patched)
> > 
> >
> > Will this set the value only if the property does not exist so that 
> > user supplied value is not over-written?

Just fixed it, should only set the property value if not already present so as 
to not overwrite an existing custom value.


- Alejandro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60919/#review180732
---


On July 17, 2017, 9:31 p.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60919/
> ---
> 
> (Updated July 17, 2017, 9:31 p.m.)
> 
> 
> Review request for Ambari, Andrii Tkach, Dmytro Grinenko, Jonathan Hurley, 
> Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-21502
> https://issues.apache.org/jira/browse/AMBARI-21502
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> During cross stack upgrade from BigInsights, if user has not set 
> custom.hive.warehouse.mode in hive-site, as the default value is changed in 
> HDFS from BigInsights default (0770) to HDP default (0777).
> EU should set the value of custom.hive.warehouse.mode to 0770
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/config-upgrade.xml
>  f1a954e 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
>  5f1e06c 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/config-upgrade.xml
>  310e504 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
>  5b8f8d9 
> 
> 
> Diff: https://reviews.apache.org/r/60919/diff/2/
> 
> 
> Testing
> ---
> 
> Python unit tests passed.
> 
> --
> Total run:1161
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Re: Review Request 60919: AMBARI-21502. Cross-stack migration from BigInsights to HDP, EU needs to set hive-site custom.hive.warehouse.mode to 0770

[Alejandro Fernandez]

> On July 17, 2017, 7:39 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/config-upgrade.xml
> > Lines 175-178 (patched)
> > 
> >
> > The descriptions says that "if the user has not set 
> > custom.hive.warehouse.mode in hive-site" ... But this directive is setting 
> > it no matter what, even if it's been configured. 
> > 
> > Should this instead use the if-present attributes?

Good catch, just fixed it.


- Alejandro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60919/#review180727
---


On July 17, 2017, 9:31 p.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60919/
> ---
> 
> (Updated July 17, 2017, 9:31 p.m.)
> 
> 
> Review request for Ambari, Andrii Tkach, Dmytro Grinenko, Jonathan Hurley, 
> Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-21502
> https://issues.apache.org/jira/browse/AMBARI-21502
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> During cross stack upgrade from BigInsights, if user has not set 
> custom.hive.warehouse.mode in hive-site, as the default value is changed in 
> HDFS from BigInsights default (0770) to HDP default (0777).
> EU should set the value of custom.hive.warehouse.mode to 0770
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/config-upgrade.xml
>  f1a954e 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
>  5f1e06c 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/config-upgrade.xml
>  310e504 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
>  5b8f8d9 
> 
> 
> Diff: https://reviews.apache.org/r/60919/diff/2/
> 
> 
> Testing
> ---
> 
> Python unit tests passed.
> 
> --
> Total run:1161
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Re: Review Request 60919: AMBARI-21502. Cross-stack migration from BigInsights to HDP, EU needs to set hive-site custom.hive.warehouse.mode to 0770

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60919/
---

(Updated July 17, 2017, 9:31 p.m.)


Review request for Ambari, Andrii Tkach, Dmytro Grinenko, Jonathan Hurley, 
Sumit Mohanty, and Sid Wagle.


Bugs: AMBARI-21502
https://issues.apache.org/jira/browse/AMBARI-21502


Repository: ambari


Description
---

During cross stack upgrade from BigInsights, if user has not set 
custom.hive.warehouse.mode in hive-site, as the default value is changed in 
HDFS from BigInsights default (0770) to HDP default (0777).
EU should set the value of custom.hive.warehouse.mode to 0770


Diffs (updated)
-

  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/config-upgrade.xml
 f1a954e 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
 5f1e06c 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/config-upgrade.xml
 310e504 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
 5b8f8d9 


Diff: https://reviews.apache.org/r/60919/diff/2/

Changes: https://reviews.apache.org/r/60919/diff/1-2/


Testing
---

Python unit tests passed.

--
Total run:1161
Total errors:0
Total failures:0
OK


Thanks,

Alejandro Fernandez

Re: Review Request 60886: AMBARI-21481. Upgrading IOP cluster with Spark2 to Ambari 2.5.2 fails on start because config mapping spark2-javaopts-properties is never selected

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60886/
---

(Updated July 17, 2017, 9:26 p.m.)


Review request for Ambari, Andrii Tkach, Dmytro Grinenko, Jonathan Hurley, 
Sumit Mohanty, and Sid Wagle.


Bugs: AMBARI-21481
https://issues.apache.org/jira/browse/AMBARI-21481


Repository: ambari


Description
---

STR:

* Install BigInsights 4.2.5 with Spark2
* Upgrade to Ambari 2.5.2
* Start Ambari, which will fail in the DB consistency check

```
2017-07-07 01:07:01,881 ERROR - You have non selected configs: 
spark2-javaopts-properties for service SPARK2 from cluster c1!
```

Looks like this config is never selected,

```
ambari=> SELECT sc.service_name, sc.version, sc.user_name, 
TO_TIMESTAMP(sc.create_timestamp/1000), sc.stack_id, sc.note,
c.version AS type_version, c.version_tag, c.type_name, c.stack_id, 
TO_TIMESTAMP(c.create_timestamp/1000) AS clusterconfig_created,
TO_TIMESTAMP(ccm.create_timestamp/1000) AS clusterconfigmapping_created, 
ccm.selected, ccm.user_name
FROM serviceconfig sc JOIN serviceconfigmapping m ON sc.service_config_id = 
m.service_config_id
JOIN clusterconfig c ON m.config_id = c.config_id
JOIN clusterconfigmapping ccm ON c.type_name = ccm.type_name AND c.version_tag 
= ccm.version_tag
WHERE sc.service_name = 'SPARK2' and c.type_name = 'spark2-javaopts-properties'
ORDER BY sc.version, c.type_name;
 service_name | version |   user_name|  to_timestamp  | stack_id |  
 note| type_version | version_tag | 
type_name  | stack_id | clusterconfig_created  | 
clusterconfigmapping_created | selected | user_name
--+-+++--+---+--+-++--++--+--+---
 SPARK2   |   1 | admin  | 2017-07-11 20:17:01+00 |3 | 
Initial configurations for Spark2 |1 | version1| 
spark2-javaopts-properties |3 | 2017-07-11 20:17:00+00 | 2017-07-11 
20:17:01+00   |0 | admin
 SPARK2   |   2 | ambari-upgrade | 2017-07-13 20:33:50+00 |3 |  
 |1 | version1| 
spark2-javaopts-properties |3 | 2017-07-11 20:17:00+00 | 2017-07-11 
20:17:01+00   |0 | admin
(2 rows)
```

This is because the config type has a single config with an empty value, so 
perhaps UI never selects it.
```

  
content
Spark2-javaopts-properties
 

  

```

Fix will be for Ambari 2.5.2 upgrade to select exactly once config for each 
config type if none are selected.
The only config type to hit this issue is spark2-javaopts-properties


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog252.java
 3c8686c 
  
ambari-server/src/main/resources/stacks/BigInsights/4.0/services/SPARK/configuration/spark-javaopts-properties.xml
 a197e34 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SPARK2/configuration/spark2-javaopts-properties.xml
 f8d50fc 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SPARK2/metainfo.xml
 bf75f47 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2/services/SPARK/configuration/spark-javaopts-properties.xml
 77a7282 


Diff: https://reviews.apache.org/r/60886/diff/5/

Changes: https://reviews.apache.org/r/60886/diff/4-5/


Testing
---

Verified on local cluster. Unit test for UpgradeCatalog252Test.java passed.


Thanks,

Alejandro Fernandez

Re: Review Request 60919: AMBARI-21502. Cross-stack migration from BigInsights to HDP, EU needs to set hive-site custom.hive.warehouse.mode to 0770

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60919/#review180732
---




ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/config-upgrade.xml
Lines 177 (patched)


Will this set the value only if the property does not exist so that user 
supplied value is not over-written?


- Sumit Mohanty


On July 17, 2017, 7:36 p.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60919/
> ---
> 
> (Updated July 17, 2017, 7:36 p.m.)
> 
> 
> Review request for Ambari, Andrii Tkach, Dmytro Grinenko, Jonathan Hurley, 
> Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-21502
> https://issues.apache.org/jira/browse/AMBARI-21502
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> During cross stack upgrade from BigInsights, if user has not set 
> custom.hive.warehouse.mode in hive-site, as the default value is changed in 
> HDFS from BigInsights default (0770) to HDP default (0777).
> EU should set the value of custom.hive.warehouse.mode to 0770
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/config-upgrade.xml
>  f1a954e 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
>  5f1e06c 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/config-upgrade.xml
>  310e504 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
>  5b8f8d9 
> 
> 
> Diff: https://reviews.apache.org/r/60919/diff/1/
> 
> 
> Testing
> ---
> 
> Python unit tests passed.
> 
> --
> Total run:1161
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Review Request 60921: Restart of MR2 History Server failed due to null in immutable_paths

[Vitalyi Brodetskyi]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60921/
---

Review request for Ambari, Andrew Onischuk, Sumit Mohanty, and Sid Wagle.


Bugs: AMBARI-21504
https://issues.apache.org/jira/browse/AMBARI-21504


Repository: ambari


Description
---

Steps:

* Installed Ambari 2.2 with Slider and services it required
* Upgraded Ambari to 2.5.2.0-134
* Registered HDP 2.6.1.0 repo, installed packages
* Restarted services that needed restart
* Ran service checks
* Started upgrade

Result: _Restarting History Server_ step failed with 

{noformat:title=errors-91.txt}
Traceback (most recent call last):
  File 
"/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py",
 line 134, in 
HistoryServer().execute()
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
 line 329, in execute
method(env)
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
 line 841, in restart
self.pre_upgrade_restart(env, upgrade_type=upgrade_type)
  File 
"/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py",
 line 85, in pre_upgrade_restart
copy_to_hdfs("mapreduce", params.user_group, params.hdfs_user, 
skip=params.sysprep_skip_copy_tarballs_hdfs)
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/functions/copy_tarball.py",
 line 256, in copy_to_hdfs
mode=0555
  File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", 
line 155, in __init__
self.env.run()
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
line 160, in run
self.run_action(resource, action)
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
line 124, in run_action
provider_action()
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py",
 line 560, in action_create_on_execute
self.action_delayed("create")
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py",
 line 550, in action_delayed
parsed_not_managed_paths = [HdfsResourceProvider.parse_path(path) for path 
in self.resource.immutable_paths]
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py",
 line 519, in parse_path
math_with_protocol_and_nn_url = re.match("[a-zA-Z]+://[^/]+(/.+)", path)
  File "/usr/lib64/python2.7/re.py", line 137, in match
return _compile(pattern, flags).match(string)
TypeError: expected string or buffer
{noformat}

{noformat:title=command-91.json}
"hostLevelParams": {
...
"not_managed_hdfs_path_list": "[null,\"/mr-history/done\",\"/app-logs\"]",
...
}
{noformat}

Failure seems to be due to {{null}} value in the list.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java
 9b19ade 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
 82171f6 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClientConfigResourceProvider.java
 daca2d0 
  ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 
ab8026c 
  
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py
 9a9f9ca 
  
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/service.py
 6495209 
  
ambari-server/src/test/java/org/apache/ambari/server/state/ConfigHelperTest.java
 b1c10f5 


Diff: https://reviews.apache.org/r/60921/diff/1/


Testing
---

mvn clean test


Thanks,

Vitalyi Brodetskyi

Re: Review Request 60919: AMBARI-21502. Cross-stack migration from BigInsights to HDP, EU needs to set hive-site custom.hive.warehouse.mode to 0770

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60919/#review180727
---




ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/config-upgrade.xml
Lines 175-178 (patched)


The descriptions says that "if the user has not set 
custom.hive.warehouse.mode in hive-site" ... But this directive is setting it 
no matter what, even if it's been configured. 

Should this instead use the if-present attributes?


- Jonathan Hurley


On July 17, 2017, 3:36 p.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60919/
> ---
> 
> (Updated July 17, 2017, 3:36 p.m.)
> 
> 
> Review request for Ambari, Andrii Tkach, Dmytro Grinenko, Jonathan Hurley, 
> Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-21502
> https://issues.apache.org/jira/browse/AMBARI-21502
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> During cross stack upgrade from BigInsights, if user has not set 
> custom.hive.warehouse.mode in hive-site, as the default value is changed in 
> HDFS from BigInsights default (0770) to HDP default (0777).
> EU should set the value of custom.hive.warehouse.mode to 0770
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/config-upgrade.xml
>  f1a954e 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
>  5f1e06c 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/config-upgrade.xml
>  310e504 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
>  5b8f8d9 
> 
> 
> Diff: https://reviews.apache.org/r/60919/diff/1/
> 
> 
> Testing
> ---
> 
> Python unit tests passed.
> 
> --
> Total run:1161
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Re: Review Request 60886: AMBARI-21481. Upgrading IOP cluster with Spark2 to Ambari 2.5.2 fails on start because config mapping spark2-javaopts-properties is never selected

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60886/#review180724
---


Fix it, then Ship it!





ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog252.java
Lines 218-224 (patched)


Can we change this so that it doesn't mention stack upgrades or HDP/BI? The 
problem is on Ambari upgrade, right? And it could happen for any config?



ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog252.java
Lines 228 (patched)


Can we do away with this check? I don't think we'd be upgrading without any 
clusters :)



ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog252.java
Lines 230 (patched)


This should probably log the types we're enforcing now, which seems to be 
only 1.


- Jonathan Hurley


On July 17, 2017, 3:15 p.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60886/
> ---
> 
> (Updated July 17, 2017, 3:15 p.m.)
> 
> 
> Review request for Ambari, Andrii Tkach, Dmytro Grinenko, Jonathan Hurley, 
> Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-21481
> https://issues.apache.org/jira/browse/AMBARI-21481
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> STR:
> 
> * Install BigInsights 4.2.5 with Spark2
> * Upgrade to Ambari 2.5.2
> * Start Ambari, which will fail in the DB consistency check
> 
> ```
> 2017-07-07 01:07:01,881 ERROR - You have non selected configs: 
> spark2-javaopts-properties for service SPARK2 from cluster c1!
> ```
> 
> Looks like this config is never selected,
> 
> ```
> ambari=> SELECT sc.service_name, sc.version, sc.user_name, 
> TO_TIMESTAMP(sc.create_timestamp/1000), sc.stack_id, sc.note,
> c.version AS type_version, c.version_tag, c.type_name, c.stack_id, 
> TO_TIMESTAMP(c.create_timestamp/1000) AS clusterconfig_created,
> TO_TIMESTAMP(ccm.create_timestamp/1000) AS clusterconfigmapping_created, 
> ccm.selected, ccm.user_name
> FROM serviceconfig sc JOIN serviceconfigmapping m ON sc.service_config_id = 
> m.service_config_id
> JOIN clusterconfig c ON m.config_id = c.config_id
> JOIN clusterconfigmapping ccm ON c.type_name = ccm.type_name AND 
> c.version_tag = ccm.version_tag
> WHERE sc.service_name = 'SPARK2' and c.type_name = 
> 'spark2-javaopts-properties'
> ORDER BY sc.version, c.type_name;
>  service_name | version |   user_name|  to_timestamp  | stack_id 
> |   note| type_version | version_tag | 
> type_name  | stack_id | clusterconfig_created  | 
> clusterconfigmapping_created | selected | user_name
> --+-+++--+---+--+-++--++--+--+---
>  SPARK2   |   1 | admin  | 2017-07-11 20:17:01+00 |3 
> | Initial configurations for Spark2 |1 | version1| 
> spark2-javaopts-properties |3 | 2017-07-11 20:17:00+00 | 2017-07-11 
> 20:17:01+00   |0 | admin
>  SPARK2   |   2 | ambari-upgrade | 2017-07-13 20:33:50+00 |3 
> |   |1 | version1| 
> spark2-javaopts-properties |3 | 2017-07-11 20:17:00+00 | 2017-07-11 
> 20:17:01+00   |0 | admin
> (2 rows)
> ```
> 
> This is because the config type has a single config with an empty value, so 
> perhaps UI never selects it.
> ```
> 
>   
> content
> Spark2-javaopts-properties
>  
> 
>   
> 
> ```
> 
> Fix will be for Ambari 2.5.2 upgrade to select exactly once config for each 
> config type if none are selected.
> The only config type to hit this issue is spark2-javaopts-properties
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog252.java
>  3c8686c 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/SPARK/configuration/spark-javaopts-properties.xml
>  a197e34 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SPARK2/configuration/spark2-javaopts-properties.xml
>  f8d50fc 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SPARK2/metainfo.xml
>  bf75f47 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2/services/SPARK/configuration/spark-javaopts-properties.xml
>  77a7282 
> 
> 
> Diff: https://reviews.apache.org/r/60886/diff/4/
> 
> 
> Testing
> ---
> 
> Verified on local 

Re: Review Request 60919: AMBARI-21502. Cross-stack migration from BigInsights to HDP, EU needs to set hive-site custom.hive.warehouse.mode to 0770

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60919/#review180725
---




ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/config-upgrade.xml
Line 137 (original), 137 (patched)


The id here was wrong, so just dropped the last 2 digits.



ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/config-upgrade.xml
Line 170 (original), 170 (patched)


The id here was wrong, so just dropped the last 2 digits.


- Alejandro Fernandez


On July 17, 2017, 7:36 p.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60919/
> ---
> 
> (Updated July 17, 2017, 7:36 p.m.)
> 
> 
> Review request for Ambari, Andrii Tkach, Dmytro Grinenko, Jonathan Hurley, 
> Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-21502
> https://issues.apache.org/jira/browse/AMBARI-21502
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> During cross stack upgrade from BigInsights, if user has not set 
> custom.hive.warehouse.mode in hive-site, as the default value is changed in 
> HDFS from BigInsights default (0770) to HDP default (0777).
> EU should set the value of custom.hive.warehouse.mode to 0770
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/config-upgrade.xml
>  f1a954e 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
>  5f1e06c 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/config-upgrade.xml
>  310e504 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
>  5b8f8d9 
> 
> 
> Diff: https://reviews.apache.org/r/60919/diff/1/
> 
> 
> Testing
> ---
> 
> Python unit tests passed.
> 
> --
> Total run:1161
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Review Request 60919: AMBARI-21502. Cross-stack migration from BigInsights to HDP, EU needs to set hive-site custom.hive.warehouse.mode to 0770

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60919/
---

Review request for Ambari, Andrii Tkach, Dmytro Grinenko, Jonathan Hurley, 
Sumit Mohanty, and Sid Wagle.


Bugs: AMBARI-21502
https://issues.apache.org/jira/browse/AMBARI-21502


Repository: ambari


Description
---

During cross stack upgrade from BigInsights, if user has not set 
custom.hive.warehouse.mode in hive-site, as the default value is changed in 
HDFS from BigInsights default (0770) to HDP default (0777).
EU should set the value of custom.hive.warehouse.mode to 0770


Diffs
-

  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/config-upgrade.xml
 f1a954e 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
 5f1e06c 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/config-upgrade.xml
 310e504 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
 5b8f8d9 


Diff: https://reviews.apache.org/r/60919/diff/1/


Testing
---

Python unit tests passed.

--
Total run:1161
Total errors:0
Total failures:0
OK


Thanks,

Alejandro Fernandez

Re: Review Request 60886: AMBARI-21481. Upgrading IOP cluster with Spark2 to Ambari 2.5.2 fails on start because config mapping spark2-javaopts-properties is never selected

[Alejandro Fernandez]

> On July 17, 2017, 6:26 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SPARK2/metainfo.xml
> > Line 82 (original), 82 (patched)
> > 
> >
> > Why indented?

Fixed


- Alejandro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60886/#review180710
---


On July 17, 2017, 7:15 p.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60886/
> ---
> 
> (Updated July 17, 2017, 7:15 p.m.)
> 
> 
> Review request for Ambari, Andrii Tkach, Dmytro Grinenko, Jonathan Hurley, 
> Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-21481
> https://issues.apache.org/jira/browse/AMBARI-21481
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> STR:
> 
> * Install BigInsights 4.2.5 with Spark2
> * Upgrade to Ambari 2.5.2
> * Start Ambari, which will fail in the DB consistency check
> 
> ```
> 2017-07-07 01:07:01,881 ERROR - You have non selected configs: 
> spark2-javaopts-properties for service SPARK2 from cluster c1!
> ```
> 
> Looks like this config is never selected,
> 
> ```
> ambari=> SELECT sc.service_name, sc.version, sc.user_name, 
> TO_TIMESTAMP(sc.create_timestamp/1000), sc.stack_id, sc.note,
> c.version AS type_version, c.version_tag, c.type_name, c.stack_id, 
> TO_TIMESTAMP(c.create_timestamp/1000) AS clusterconfig_created,
> TO_TIMESTAMP(ccm.create_timestamp/1000) AS clusterconfigmapping_created, 
> ccm.selected, ccm.user_name
> FROM serviceconfig sc JOIN serviceconfigmapping m ON sc.service_config_id = 
> m.service_config_id
> JOIN clusterconfig c ON m.config_id = c.config_id
> JOIN clusterconfigmapping ccm ON c.type_name = ccm.type_name AND 
> c.version_tag = ccm.version_tag
> WHERE sc.service_name = 'SPARK2' and c.type_name = 
> 'spark2-javaopts-properties'
> ORDER BY sc.version, c.type_name;
>  service_name | version |   user_name|  to_timestamp  | stack_id 
> |   note| type_version | version_tag | 
> type_name  | stack_id | clusterconfig_created  | 
> clusterconfigmapping_created | selected | user_name
> --+-+++--+---+--+-++--++--+--+---
>  SPARK2   |   1 | admin  | 2017-07-11 20:17:01+00 |3 
> | Initial configurations for Spark2 |1 | version1| 
> spark2-javaopts-properties |3 | 2017-07-11 20:17:00+00 | 2017-07-11 
> 20:17:01+00   |0 | admin
>  SPARK2   |   2 | ambari-upgrade | 2017-07-13 20:33:50+00 |3 
> |   |1 | version1| 
> spark2-javaopts-properties |3 | 2017-07-11 20:17:00+00 | 2017-07-11 
> 20:17:01+00   |0 | admin
> (2 rows)
> ```
> 
> This is because the config type has a single config with an empty value, so 
> perhaps UI never selects it.
> ```
> 
>   
> content
> Spark2-javaopts-properties
>  
> 
>   
> 
> ```
> 
> Fix will be for Ambari 2.5.2 upgrade to select exactly once config for each 
> config type if none are selected.
> The only config type to hit this issue is spark2-javaopts-properties
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog252.java
>  3c8686c 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/SPARK/configuration/spark-javaopts-properties.xml
>  a197e34 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SPARK2/configuration/spark2-javaopts-properties.xml
>  f8d50fc 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SPARK2/metainfo.xml
>  bf75f47 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2/services/SPARK/configuration/spark-javaopts-properties.xml
>  77a7282 
> 
> 
> Diff: https://reviews.apache.org/r/60886/diff/4/
> 
> 
> Testing
> ---
> 
> Verified on local cluster. Unit test for UpgradeCatalog252Test.java passed.
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Re: Review Request 60886: AMBARI-21481. Upgrading IOP cluster with Spark2 to Ambari 2.5.2 fails on start because config mapping spark2-javaopts-properties is never selected

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60886/
---

(Updated July 17, 2017, 7:15 p.m.)


Review request for Ambari, Andrii Tkach, Dmytro Grinenko, Jonathan Hurley, 
Sumit Mohanty, and Sid Wagle.


Bugs: AMBARI-21481
https://issues.apache.org/jira/browse/AMBARI-21481


Repository: ambari


Description
---

STR:

* Install BigInsights 4.2.5 with Spark2
* Upgrade to Ambari 2.5.2
* Start Ambari, which will fail in the DB consistency check

```
2017-07-07 01:07:01,881 ERROR - You have non selected configs: 
spark2-javaopts-properties for service SPARK2 from cluster c1!
```

Looks like this config is never selected,

```
ambari=> SELECT sc.service_name, sc.version, sc.user_name, 
TO_TIMESTAMP(sc.create_timestamp/1000), sc.stack_id, sc.note,
c.version AS type_version, c.version_tag, c.type_name, c.stack_id, 
TO_TIMESTAMP(c.create_timestamp/1000) AS clusterconfig_created,
TO_TIMESTAMP(ccm.create_timestamp/1000) AS clusterconfigmapping_created, 
ccm.selected, ccm.user_name
FROM serviceconfig sc JOIN serviceconfigmapping m ON sc.service_config_id = 
m.service_config_id
JOIN clusterconfig c ON m.config_id = c.config_id
JOIN clusterconfigmapping ccm ON c.type_name = ccm.type_name AND c.version_tag 
= ccm.version_tag
WHERE sc.service_name = 'SPARK2' and c.type_name = 'spark2-javaopts-properties'
ORDER BY sc.version, c.type_name;
 service_name | version |   user_name|  to_timestamp  | stack_id |  
 note| type_version | version_tag | 
type_name  | stack_id | clusterconfig_created  | 
clusterconfigmapping_created | selected | user_name
--+-+++--+---+--+-++--++--+--+---
 SPARK2   |   1 | admin  | 2017-07-11 20:17:01+00 |3 | 
Initial configurations for Spark2 |1 | version1| 
spark2-javaopts-properties |3 | 2017-07-11 20:17:00+00 | 2017-07-11 
20:17:01+00   |0 | admin
 SPARK2   |   2 | ambari-upgrade | 2017-07-13 20:33:50+00 |3 |  
 |1 | version1| 
spark2-javaopts-properties |3 | 2017-07-11 20:17:00+00 | 2017-07-11 
20:17:01+00   |0 | admin
(2 rows)
```

This is because the config type has a single config with an empty value, so 
perhaps UI never selects it.
```

  
content
Spark2-javaopts-properties
 

  

```

Fix will be for Ambari 2.5.2 upgrade to select exactly once config for each 
config type if none are selected.
The only config type to hit this issue is spark2-javaopts-properties


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog252.java
 3c8686c 
  
ambari-server/src/main/resources/stacks/BigInsights/4.0/services/SPARK/configuration/spark-javaopts-properties.xml
 a197e34 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SPARK2/configuration/spark2-javaopts-properties.xml
 f8d50fc 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SPARK2/metainfo.xml
 bf75f47 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2/services/SPARK/configuration/spark-javaopts-properties.xml
 77a7282 


Diff: https://reviews.apache.org/r/60886/diff/4/

Changes: https://reviews.apache.org/r/60886/diff/3-4/


Testing
---

Verified on local cluster. Unit test for UpgradeCatalog252Test.java passed.


Thanks,

Alejandro Fernandez

Re: Review Request 60886: AMBARI-21481. Upgrading IOP cluster with Spark2 to Ambari 2.5.2 fails on start because config mapping spark2-javaopts-properties is never selected

[Alejandro Fernandez]

> On July 17, 2017, 6:26 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog252.java
> > Lines 214-221 (patched)
> > 
> >
> > I don't think this comment is correct - this doesn't deal with a stack 
> > upgrade - it deals with upgrading Ambari, right?
> > 
> > Also - what happens with services which were deleted. Their 
> > configurations remain unselected, no?

This issue is due to Ambari Upgrade, which fails starting the server due to the 
DB consistency checks we have in Ambari 2.5.2


- Alejandro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60886/#review180710
---


On July 17, 2017, 7:12 p.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60886/
> ---
> 
> (Updated July 17, 2017, 7:12 p.m.)
> 
> 
> Review request for Ambari, Andrii Tkach, Dmytro Grinenko, Jonathan Hurley, 
> Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-21481
> https://issues.apache.org/jira/browse/AMBARI-21481
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> STR:
> 
> * Install BigInsights 4.2.5 with Spark2
> * Upgrade to Ambari 2.5.2
> * Start Ambari, which will fail in the DB consistency check
> 
> ```
> 2017-07-07 01:07:01,881 ERROR - You have non selected configs: 
> spark2-javaopts-properties for service SPARK2 from cluster c1!
> ```
> 
> Looks like this config is never selected,
> 
> ```
> ambari=> SELECT sc.service_name, sc.version, sc.user_name, 
> TO_TIMESTAMP(sc.create_timestamp/1000), sc.stack_id, sc.note,
> c.version AS type_version, c.version_tag, c.type_name, c.stack_id, 
> TO_TIMESTAMP(c.create_timestamp/1000) AS clusterconfig_created,
> TO_TIMESTAMP(ccm.create_timestamp/1000) AS clusterconfigmapping_created, 
> ccm.selected, ccm.user_name
> FROM serviceconfig sc JOIN serviceconfigmapping m ON sc.service_config_id = 
> m.service_config_id
> JOIN clusterconfig c ON m.config_id = c.config_id
> JOIN clusterconfigmapping ccm ON c.type_name = ccm.type_name AND 
> c.version_tag = ccm.version_tag
> WHERE sc.service_name = 'SPARK2' and c.type_name = 
> 'spark2-javaopts-properties'
> ORDER BY sc.version, c.type_name;
>  service_name | version |   user_name|  to_timestamp  | stack_id 
> |   note| type_version | version_tag | 
> type_name  | stack_id | clusterconfig_created  | 
> clusterconfigmapping_created | selected | user_name
> --+-+++--+---+--+-++--++--+--+---
>  SPARK2   |   1 | admin  | 2017-07-11 20:17:01+00 |3 
> | Initial configurations for Spark2 |1 | version1| 
> spark2-javaopts-properties |3 | 2017-07-11 20:17:00+00 | 2017-07-11 
> 20:17:01+00   |0 | admin
>  SPARK2   |   2 | ambari-upgrade | 2017-07-13 20:33:50+00 |3 
> |   |1 | version1| 
> spark2-javaopts-properties |3 | 2017-07-11 20:17:00+00 | 2017-07-11 
> 20:17:01+00   |0 | admin
> (2 rows)
> ```
> 
> This is because the config type has a single config with an empty value, so 
> perhaps UI never selects it.
> ```
> 
>   
> content
> Spark2-javaopts-properties
>  
> 
>   
> 
> ```
> 
> Fix will be for Ambari 2.5.2 upgrade to select exactly once config for each 
> config type if none are selected.
> The only config type to hit this issue is spark2-javaopts-properties
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog252.java
>  3c8686c 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/SPARK/configuration/spark-javaopts-properties.xml
>  a197e34 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SPARK2/configuration/spark2-javaopts-properties.xml
>  f8d50fc 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SPARK2/metainfo.xml
>  bf75f47 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2/services/SPARK/configuration/spark-javaopts-properties.xml
>  77a7282 
> 
> 
> Diff: https://reviews.apache.org/r/60886/diff/3/
> 
> 
> Testing
> ---
> 
> Verified on local cluster. Unit test for UpgradeCatalog252Test.java passed.
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Re: Review Request 60886: AMBARI-21481. Upgrading IOP cluster with Spark2 to Ambari 2.5.2 fails on start because config mapping spark2-javaopts-properties is never selected

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60886/
---

(Updated July 17, 2017, 7:12 p.m.)


Review request for Ambari, Andrii Tkach, Dmytro Grinenko, Jonathan Hurley, 
Sumit Mohanty, and Sid Wagle.


Bugs: AMBARI-21481
https://issues.apache.org/jira/browse/AMBARI-21481


Repository: ambari


Description
---

STR:

* Install BigInsights 4.2.5 with Spark2
* Upgrade to Ambari 2.5.2
* Start Ambari, which will fail in the DB consistency check

```
2017-07-07 01:07:01,881 ERROR - You have non selected configs: 
spark2-javaopts-properties for service SPARK2 from cluster c1!
```

Looks like this config is never selected,

```
ambari=> SELECT sc.service_name, sc.version, sc.user_name, 
TO_TIMESTAMP(sc.create_timestamp/1000), sc.stack_id, sc.note,
c.version AS type_version, c.version_tag, c.type_name, c.stack_id, 
TO_TIMESTAMP(c.create_timestamp/1000) AS clusterconfig_created,
TO_TIMESTAMP(ccm.create_timestamp/1000) AS clusterconfigmapping_created, 
ccm.selected, ccm.user_name
FROM serviceconfig sc JOIN serviceconfigmapping m ON sc.service_config_id = 
m.service_config_id
JOIN clusterconfig c ON m.config_id = c.config_id
JOIN clusterconfigmapping ccm ON c.type_name = ccm.type_name AND c.version_tag 
= ccm.version_tag
WHERE sc.service_name = 'SPARK2' and c.type_name = 'spark2-javaopts-properties'
ORDER BY sc.version, c.type_name;
 service_name | version |   user_name|  to_timestamp  | stack_id |  
 note| type_version | version_tag | 
type_name  | stack_id | clusterconfig_created  | 
clusterconfigmapping_created | selected | user_name
--+-+++--+---+--+-++--++--+--+---
 SPARK2   |   1 | admin  | 2017-07-11 20:17:01+00 |3 | 
Initial configurations for Spark2 |1 | version1| 
spark2-javaopts-properties |3 | 2017-07-11 20:17:00+00 | 2017-07-11 
20:17:01+00   |0 | admin
 SPARK2   |   2 | ambari-upgrade | 2017-07-13 20:33:50+00 |3 |  
 |1 | version1| 
spark2-javaopts-properties |3 | 2017-07-11 20:17:00+00 | 2017-07-11 
20:17:01+00   |0 | admin
(2 rows)
```

This is because the config type has a single config with an empty value, so 
perhaps UI never selects it.
```

  
content
Spark2-javaopts-properties
 

  

```

Fix will be for Ambari 2.5.2 upgrade to select exactly once config for each 
config type if none are selected.
The only config type to hit this issue is spark2-javaopts-properties


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog252.java
 3c8686c 
  
ambari-server/src/main/resources/stacks/BigInsights/4.0/services/SPARK/configuration/spark-javaopts-properties.xml
 a197e34 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SPARK2/configuration/spark2-javaopts-properties.xml
 f8d50fc 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SPARK2/metainfo.xml
 bf75f47 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2/services/SPARK/configuration/spark-javaopts-properties.xml
 77a7282 


Diff: https://reviews.apache.org/r/60886/diff/3/

Changes: https://reviews.apache.org/r/60886/diff/2-3/


Testing
---

Verified on local cluster. Unit test for UpgradeCatalog252Test.java passed.


Thanks,

Alejandro Fernandez

Re: Review Request 60916: Support SPARK2 upgrade for BI 4.2.5 to HDP 2.6.2

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60916/#review180712
---


Ship it!




Ship It!

- Alejandro Fernandez


On July 17, 2017, 5:40 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60916/
> ---
> 
> (Updated July 17, 2017, 5:40 p.m.)
> 
> 
> Review request for Ambari, Attila Doroszlai and Sid Wagle.
> 
> 
> Bugs: AMBARI-21500
> https://issues.apache.org/jira/browse/AMBARI-21500
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> SPARK2 is missing from the upgrade pack for BI 4.2.5 resulting in finalize 
> operation error:
> 
> {code}
> The following components were found to have version mismatches.  Finalize 
> will not complete successfully:
> c7302.ambari.apache.org: SPARK2/SPARK2_THRIFTSERVER reports 4.2.5.0-
> c7302.ambari.apache.org: SPARK2/SPARK2_CLIENT reports 4.2.5.0-
> c7302.ambari.apache.org: SPARK2/SPARK2_JOBHISTORYSERVER reports 4.2.5.0-
> {code}
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/config-upgrade.xml
>  f1a954ea57 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
>  5f1e06c9ba 
> 
> 
> Diff: https://reviews.apache.org/r/60916/diff/1/
> 
> 
> Testing
> ---
> 
> PENDING...
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Re: Review Request 60912: Fix HDP base services to handle changed config paths

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60912/#review180711
---


Ship it!




Ship It!

- Alejandro Fernandez


On July 17, 2017, 2:17 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60912/
> ---
> 
> (Updated July 17, 2017, 2:17 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Myroslav Papirkovskyy, and 
> Sid Wagle.
> 
> 
> Bugs: AMBARI-21496
> https://issues.apache.org/jira/browse/AMBARI-21496
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ACCUMULO/1.6.1.2.2.0/package/scripts/accumulo_configuration.py
>  f65b68a 
>   
> ambari-server/src/main/resources/common-services/ACCUMULO/1.6.1.2.2.0/package/scripts/params.py
>  53cf002 
>   
> ambari-server/src/main/resources/common-services/ACCUMULO/1.6.1.2.2.0/package/scripts/status_params.py
>  9fc89f1 
>   
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/params.py
>  acf420e 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/ams.py
>  a929847 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/hbase.py
>  e20ff1d 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py
>  919f26d 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/status_params.py
>  6057a8b 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata.py
>  36c4598 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py
>  d26df33 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/setup_ranger_atlas.py
>  c47c75c 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/status_params.py
>  852a9cb 
>   
> ambari-server/src/main/resources/common-services/DRUID/0.9.2/package/scripts/params.py
>  b1d9472 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/params.py
>  2942992 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/params_linux.py
>  3773918 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/status_params.py
>  0a0e370 
>   
> ambari-server/src/main/resources/common-services/FLUME/1.4.0.2.0/package/scripts/params.py
>  b143941 
>   
> ambari-server/src/main/resources/common-services/FLUME/1.4.0.2.0/package/scripts/params_linux.py
>  53b6be0 
>   
> ambari-server/src/main/resources/common-services/GANGLIA/3.5.0/package/scripts/params.py
>  8473013 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/params.py
>  e2dbd7b 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/hbase.py
>  8ad802e 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py
>  3177643 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/setup_ranger_hbase.py
>  d32dce1 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/status_params.py
>  bc75c78 
>   
> ambari-server/src/main/resources/common-services/HBASE/2.0.0.3.0/package/scripts/hbase.py
>  fced4fc 
>   
> ambari-server/src/main/resources/common-services/HBASE/2.0.0.3.0/package/scripts/params_linux.py
>  b7e2b89 
>   
> ambari-server/src/main/resources/common-services/HBASE/2.0.0.3.0/package/scripts/setup_ranger_hbase.py
>  d32dce1 
>   
> ambari-server/src/main/resources/common-services/HBASE/2.0.0.3.0/package/scripts/status_params.py
>  bc75c78 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs.py
>  d9b62e2 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/install_params.py
>  fe488c3 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py
>  e88dbdd 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_windows.py
>  1e47c29 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/setup_ranger_hdfs.py
>  47c6e35 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/status_params.py
>  153f9a6 
>   
> ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/hdfs.py
>  d9b62e2 
>   
> ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/install_params.py
>  

Re: Review Request 60916: Support SPARK2 upgrade for BI 4.2.5 to HDP 2.6.2

[Jonathan Hurley]

> On July 17, 2017, 1:52 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
> > Lines 945 (patched)
> > 
> >
> > Why only one express upgrade pack and not both?

SPARK2 wasn't a service in BI 4.2.0


- Jonathan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60916/#review180703
---


On July 17, 2017, 1:40 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60916/
> ---
> 
> (Updated July 17, 2017, 1:40 p.m.)
> 
> 
> Review request for Ambari, Attila Doroszlai and Sid Wagle.
> 
> 
> Bugs: AMBARI-21500
> https://issues.apache.org/jira/browse/AMBARI-21500
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> SPARK2 is missing from the upgrade pack for BI 4.2.5 resulting in finalize 
> operation error:
> 
> {code}
> The following components were found to have version mismatches.  Finalize 
> will not complete successfully:
> c7302.ambari.apache.org: SPARK2/SPARK2_THRIFTSERVER reports 4.2.5.0-
> c7302.ambari.apache.org: SPARK2/SPARK2_CLIENT reports 4.2.5.0-
> c7302.ambari.apache.org: SPARK2/SPARK2_JOBHISTORYSERVER reports 4.2.5.0-
> {code}
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/config-upgrade.xml
>  f1a954ea57 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
>  5f1e06c9ba 
> 
> 
> Diff: https://reviews.apache.org/r/60916/diff/1/
> 
> 
> Testing
> ---
> 
> PENDING...
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Re: Review Request 60906: Add SOLR to BigInsights 4.2.5 stack

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60906/#review180705
---


Ship it!




Ship It!

- Alejandro Fernandez


On July 17, 2017, 12:08 p.m., Oliver Szabo wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60906/
> ---
> 
> (Updated July 17, 2017, 12:08 p.m.)
> 
> 
> Review request for Ambari, Attila Doroszlai, Jonathan Hurley, Robert 
> Nettleton, Sumit Mohanty, Sebastian Toader, and Sid Wagle.
> 
> 
> Bugs: AMBARI-21487
> https://issues.apache.org/jira/browse/AMBARI-21487
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> SOLR is missing from 4.2.5 stack. I moved the solr stack code from 
> common-services to stacks/BigInsights/4.2.5/
> 
> note: in the stack metainfo.xml there is an  tag for 4.2 stack, that 
> means a solr-site config type is added during ambari upgrade (because of 
> on-ambari-upgrade add="true" tags), because of that solr needs to be 
> restarted. if i change the on-ambari-upgrade add flag to false, its not 
> needed to restart SOLR, but because of the  of metainfo.xml, db 
> check will fail. So i kept that behaviour.
> 
> next: HDP upgrade
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/alerts.json
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/configuration/ranger-solr-audit.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/configuration/ranger-solr-plugin-properties.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/configuration/ranger-solr-policymgr-ssl.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/configuration/ranger-solr-security.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/configuration/solr-client-log4j.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/configuration/solr-env.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/configuration/solr-log4j.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/configuration/solr-xml.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/kerberos.json
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/metainfo.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/package/scripts/params.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/package/scripts/service_check.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/package/scripts/setup_ranger_solr.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/package/scripts/setup_solr.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/package/scripts/solr.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/package/scripts/solr_client.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/package/scripts/status_params.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/package/templates/solr_jaas.conf.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/properties/solr-client-log4j.properties.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/properties/solr-env.sh.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/properties/solr-log4j.properties.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/properties/solr.xml.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/quicklinks/quicklinks.json
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/themes/theme.json
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/60906/diff/1/
> 
> 
> Testing
> ---
> 
> test manually:
> Upgrade ambari -> start/stop solr.
> 
> 
> Thanks,
> 
> Oliver Szabo
> 
>

Re: Review Request 60916: Support SPARK2 upgrade for BI 4.2.5 to HDP 2.6.2

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60916/#review180703
---




ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
Lines 945 (patched)


Why only one express upgrade pack and not both?


- Alejandro Fernandez


On July 17, 2017, 5:40 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60916/
> ---
> 
> (Updated July 17, 2017, 5:40 p.m.)
> 
> 
> Review request for Ambari, Attila Doroszlai and Sid Wagle.
> 
> 
> Bugs: AMBARI-21500
> https://issues.apache.org/jira/browse/AMBARI-21500
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> SPARK2 is missing from the upgrade pack for BI 4.2.5 resulting in finalize 
> operation error:
> 
> {code}
> The following components were found to have version mismatches.  Finalize 
> will not complete successfully:
> c7302.ambari.apache.org: SPARK2/SPARK2_THRIFTSERVER reports 4.2.5.0-
> c7302.ambari.apache.org: SPARK2/SPARK2_CLIENT reports 4.2.5.0-
> c7302.ambari.apache.org: SPARK2/SPARK2_JOBHISTORYSERVER reports 4.2.5.0-
> {code}
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/config-upgrade.xml
>  f1a954ea57 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
>  5f1e06c9ba 
> 
> 
> Diff: https://reviews.apache.org/r/60916/diff/1/
> 
> 
> Testing
> ---
> 
> PENDING...
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Review Request 60916: Support SPARK2 upgrade for BI 4.2.5 to HDP 2.6.2

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60916/
---

Review request for Ambari, Attila Doroszlai and Sid Wagle.


Bugs: AMBARI-21500
https://issues.apache.org/jira/browse/AMBARI-21500


Repository: ambari


Description
---

SPARK2 is missing from the upgrade pack for BI 4.2.5 resulting in finalize 
operation error:

{code}
The following components were found to have version mismatches.  Finalize will 
not complete successfully:
c7302.ambari.apache.org: SPARK2/SPARK2_THRIFTSERVER reports 4.2.5.0-
c7302.ambari.apache.org: SPARK2/SPARK2_CLIENT reports 4.2.5.0-
c7302.ambari.apache.org: SPARK2/SPARK2_JOBHISTORYSERVER reports 4.2.5.0-
{code}


Diffs
-

  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/config-upgrade.xml
 f1a954ea57 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
 5f1e06c9ba 


Diff: https://reviews.apache.org/r/60916/diff/1/


Testing
---

PENDING...


Thanks,

Jonathan Hurley

Re: Review Request 60906: Add SOLR to BigInsights 4.2.5 stack

[Attila Doroszlai]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60906/#review180696
---


Ship it!




Ship It!

- Attila Doroszlai


On July 17, 2017, 2:08 p.m., Oliver Szabo wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60906/
> ---
> 
> (Updated July 17, 2017, 2:08 p.m.)
> 
> 
> Review request for Ambari, Attila Doroszlai, Jonathan Hurley, Robert 
> Nettleton, Sumit Mohanty, Sebastian Toader, and Sid Wagle.
> 
> 
> Bugs: AMBARI-21487
> https://issues.apache.org/jira/browse/AMBARI-21487
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> SOLR is missing from 4.2.5 stack. I moved the solr stack code from 
> common-services to stacks/BigInsights/4.2.5/
> 
> note: in the stack metainfo.xml there is an  tag for 4.2 stack, that 
> means a solr-site config type is added during ambari upgrade (because of 
> on-ambari-upgrade add="true" tags), because of that solr needs to be 
> restarted. if i change the on-ambari-upgrade add flag to false, its not 
> needed to restart SOLR, but because of the  of metainfo.xml, db 
> check will fail. So i kept that behaviour.
> 
> next: HDP upgrade
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/alerts.json
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/configuration/ranger-solr-audit.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/configuration/ranger-solr-plugin-properties.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/configuration/ranger-solr-policymgr-ssl.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/configuration/ranger-solr-security.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/configuration/solr-client-log4j.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/configuration/solr-env.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/configuration/solr-log4j.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/configuration/solr-xml.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/kerberos.json
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/metainfo.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/package/scripts/params.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/package/scripts/service_check.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/package/scripts/setup_ranger_solr.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/package/scripts/setup_solr.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/package/scripts/solr.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/package/scripts/solr_client.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/package/scripts/status_params.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/package/templates/solr_jaas.conf.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/properties/solr-client-log4j.properties.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/properties/solr-env.sh.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/properties/solr-log4j.properties.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/properties/solr.xml.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/quicklinks/quicklinks.json
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/themes/theme.json
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/60906/diff/1/
> 
> 
> Testing
> ---
> 
> test manually:
> Upgrade ambari -> start/stop solr.
> 
> 
> Thanks,
> 
> Oliver Szabo
> 
>

Re: Review Request 60910: Fix PERF stack scripts to handle changed config paths

[Sid Wagle]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60910/#review180691
---


Ship it!




Ship It!

- Sid Wagle


On July 17, 2017, 1:34 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60910/
> ---
> 
> (Updated July 17, 2017, 1:34 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Myroslav Papirkovskyy, and 
> Sid Wagle.
> 
> 
> Bugs: AMBARI-21493
> https://issues.apache.org/jira/browse/AMBARI-21493
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/get_not_managed_resources.py
>  4af636b 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py
>  2b3df5f 
>   
> ambari-common/src/main/python/resource_management/libraries/providers/xml_config.py
>  28697bf 
>   ambari-common/src/main/python/resource_management/libraries/script/dummy.py 
> ad5f2a6 
>   
> ambari-common/src/main/python/resource_management/libraries/script/script.py 
> a08feab 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/params.py
>  9abd2fe 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/shared_initialization.py
>  36a202f 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-ANY/scripts/params.py
>  4052d1d 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/params.py
>  6193c11 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/repo_initialization.py
>  a35dce7 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/params.py
>  70ebfeb 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/shared_initialization.py
>  148d235 
>   
> ambari-server/src/main/resources/stacks/HDP/3.0/hooks/after-INSTALL/scripts/params.py
>  5dcd39b 
>   
> ambari-server/src/main/resources/stacks/HDP/3.0/hooks/after-INSTALL/scripts/shared_initialization.py
>  e9f2283 
>   
> ambari-server/src/main/resources/stacks/HDP/3.0/hooks/before-ANY/scripts/params.py
>  9be9101 
>   
> ambari-server/src/main/resources/stacks/HDP/3.0/hooks/before-INSTALL/scripts/params.py
>  6193c11 
>   
> ambari-server/src/main/resources/stacks/HDP/3.0/hooks/before-START/scripts/params.py
>  5a5361c 
>   
> ambari-server/src/main/resources/stacks/PERF/1.0/hooks/before-ANY/scripts/params.py
>  2c2c901 
>   
> ambari-server/src/main/resources/stacks/PERF/1.0/services/FAKEHBASE/metainfo.xml
>  66d5a29 
>   
> ambari-server/src/main/resources/stacks/PERF/1.0/services/FAKEHDFS/metainfo.xml
>  13b10e0 
>   
> ambari-server/src/main/resources/stacks/PERF/1.0/services/FAKEHDFS/package/scripts/params.py
>  8068441 
>   
> ambari-server/src/main/resources/stacks/PERF/1.0/services/KERBEROS/package/scripts/params.py
>  4eb5b02 
> 
> 
> Diff: https://reviews.apache.org/r/60910/diff/1/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 60859: Zeppelin does not start and returns: params.zookeeper_znode_parent not in interpreter['properties']['phoenix.url']: KeyError: 'phoenix.url'

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60859/#review180690
---


Ship it!




Ship It!

- Alejandro Fernandez


On July 14, 2017, 11:18 a.m., Prabhjyot Singh wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60859/
> ---
> 
> (Updated July 14, 2017, 11:18 a.m.)
> 
> 
> Review request for Ambari, Attila Doroszlai, Alejandro Fernandez, DIPAYAN 
> BHOWMICK, Jayush Luniya, Rohit Choudhary, Renjith Kamath, Sumit Mohanty, and 
> venkat sairam.
> 
> 
> Bugs: AMBARI-21473
> https://issues.apache.org/jira/browse/AMBARI-21473
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Zeppelin does not start and returns: params.zookeeper_znode_parent not in 
> interpreter['properties']['phoenix.url']: KeyError: 'phoenix.url'
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/master.py
>  0013ab0eb8 
> 
> 
> Diff: https://reviews.apache.org/r/60859/diff/1/
> 
> 
> Testing
> ---
> 
> Manually on CentOS 6 with kerberos enabled
> 
> 
> Thanks,
> 
> Prabhjyot Singh
> 
>

Re: Review Request 60859: Zeppelin does not start and returns: params.zookeeper_znode_parent not in interpreter['properties']['phoenix.url']: KeyError: 'phoenix.url'

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60859/#review180689
---


Ship it!




Ship It!

- Alejandro Fernandez


On July 14, 2017, 11:18 a.m., Prabhjyot Singh wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60859/
> ---
> 
> (Updated July 14, 2017, 11:18 a.m.)
> 
> 
> Review request for Ambari, Attila Doroszlai, Alejandro Fernandez, DIPAYAN 
> BHOWMICK, Jayush Luniya, Rohit Choudhary, Renjith Kamath, Sumit Mohanty, and 
> venkat sairam.
> 
> 
> Bugs: AMBARI-21473
> https://issues.apache.org/jira/browse/AMBARI-21473
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Zeppelin does not start and returns: params.zookeeper_znode_parent not in 
> interpreter['properties']['phoenix.url']: KeyError: 'phoenix.url'
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/master.py
>  0013ab0eb8 
> 
> 
> Diff: https://reviews.apache.org/r/60859/diff/1/
> 
> 
> Testing
> ---
> 
> Manually on CentOS 6 with kerberos enabled
> 
> 
> Thanks,
> 
> Prabhjyot Singh
> 
>

Re: Review Request 60886: AMBARI-21481. Upgrading IOP cluster with Spark2 to Ambari 2.5.2 fails on start because config mapping spark2-javaopts-properties is never selected

[Alejandro Fernandez]

> On July 15, 2017, 12:40 a.m., Sid Wagle wrote:
> > What happens to configmappings unselected due to service delete? What 
> > wouldbe side effects of selecting those?

Before a service is deleted, it should have exactly one config mapping for a 
certain config type as selected (and in this case that condition is not met). 
If a service is deleted, I need to check if we unselect all of its config 
mappings; but either way it shouldn't matter because if the service is re-added 
we always generate a new config version.


- Alejandro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60886/#review180617
---


On July 15, 2017, 12:20 a.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60886/
> ---
> 
> (Updated July 15, 2017, 12:20 a.m.)
> 
> 
> Review request for Ambari, Andrii Tkach, Dmytro Grinenko, Jonathan Hurley, 
> Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-21481
> https://issues.apache.org/jira/browse/AMBARI-21481
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> STR:
> 
> * Install BigInsights 4.2.5 with Spark2
> * Upgrade to Ambari 2.5.2
> * Start Ambari, which will fail in the DB consistency check
> 
> ```
> 2017-07-07 01:07:01,881 ERROR - You have non selected configs: 
> spark2-javaopts-properties for service SPARK2 from cluster c1!
> ```
> 
> Looks like this config is never selected,
> 
> ```
> ambari=> SELECT sc.service_name, sc.version, sc.user_name, 
> TO_TIMESTAMP(sc.create_timestamp/1000), sc.stack_id, sc.note,
> c.version AS type_version, c.version_tag, c.type_name, c.stack_id, 
> TO_TIMESTAMP(c.create_timestamp/1000) AS clusterconfig_created,
> TO_TIMESTAMP(ccm.create_timestamp/1000) AS clusterconfigmapping_created, 
> ccm.selected, ccm.user_name
> FROM serviceconfig sc JOIN serviceconfigmapping m ON sc.service_config_id = 
> m.service_config_id
> JOIN clusterconfig c ON m.config_id = c.config_id
> JOIN clusterconfigmapping ccm ON c.type_name = ccm.type_name AND 
> c.version_tag = ccm.version_tag
> WHERE sc.service_name = 'SPARK2' and c.type_name = 
> 'spark2-javaopts-properties'
> ORDER BY sc.version, c.type_name;
>  service_name | version |   user_name|  to_timestamp  | stack_id 
> |   note| type_version | version_tag | 
> type_name  | stack_id | clusterconfig_created  | 
> clusterconfigmapping_created | selected | user_name
> --+-+++--+---+--+-++--++--+--+---
>  SPARK2   |   1 | admin  | 2017-07-11 20:17:01+00 |3 
> | Initial configurations for Spark2 |1 | version1| 
> spark2-javaopts-properties |3 | 2017-07-11 20:17:00+00 | 2017-07-11 
> 20:17:01+00   |0 | admin
>  SPARK2   |   2 | ambari-upgrade | 2017-07-13 20:33:50+00 |3 
> |   |1 | version1| 
> spark2-javaopts-properties |3 | 2017-07-11 20:17:00+00 | 2017-07-11 
> 20:17:01+00   |0 | admin
> (2 rows)
> ```
> 
> This is because the config type has a single config with an empty value, so 
> perhaps UI never selects it.
> ```
> 
>   
> content
> Spark2-javaopts-properties
>  
> 
>   
> 
> ```
> 
> Fix will be for Ambari 2.5.2 upgrade to select exactly once config for each 
> config type if none are selected.
> The only config type to hit this issue is spark2-javaopts-properties
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog252.java
>  6d2ab84 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/SPARK/configuration/spark-javaopts-properties.xml
>  a197e34 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SPARK2/configuration/spark2-javaopts-properties.xml
>  f8d50fc 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SPARK2/metainfo.xml
>  bf75f47 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2/services/SPARK/configuration/spark-javaopts-properties.xml
>  77a7282 
> 
> 
> Diff: https://reviews.apache.org/r/60886/diff/2/
> 
> 
> Testing
> ---
> 
> Verified on local cluster. Unit test for UpgradeCatalog252Test.java passed.
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Re: Review Request 60886: AMBARI-21481. Upgrading IOP cluster with Spark2 to Ambari 2.5.2 fails on start because config mapping spark2-javaopts-properties is never selected

[Alejandro Fernandez]

> On July 17, 2017, 3:29 p.m., Jonathan Hurley wrote:
> > So, this is an env file with no content? Doesn't that mean that it's 
> > optional and doesn't need to be created? Can we simply use the  
> > tag (see AMBARI-21364) and have the DB checker skip it?

The config is already saved to the DB consistency checker applies its business 
logic that at least one config mapping must be selected. If this were HDP, I 
could also mark it as  but since it's for the BigInsights stack it 
won't affect any new clusters.


- Alejandro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60886/#review180681
---


On July 15, 2017, 12:20 a.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60886/
> ---
> 
> (Updated July 15, 2017, 12:20 a.m.)
> 
> 
> Review request for Ambari, Andrii Tkach, Dmytro Grinenko, Jonathan Hurley, 
> Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-21481
> https://issues.apache.org/jira/browse/AMBARI-21481
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> STR:
> 
> * Install BigInsights 4.2.5 with Spark2
> * Upgrade to Ambari 2.5.2
> * Start Ambari, which will fail in the DB consistency check
> 
> ```
> 2017-07-07 01:07:01,881 ERROR - You have non selected configs: 
> spark2-javaopts-properties for service SPARK2 from cluster c1!
> ```
> 
> Looks like this config is never selected,
> 
> ```
> ambari=> SELECT sc.service_name, sc.version, sc.user_name, 
> TO_TIMESTAMP(sc.create_timestamp/1000), sc.stack_id, sc.note,
> c.version AS type_version, c.version_tag, c.type_name, c.stack_id, 
> TO_TIMESTAMP(c.create_timestamp/1000) AS clusterconfig_created,
> TO_TIMESTAMP(ccm.create_timestamp/1000) AS clusterconfigmapping_created, 
> ccm.selected, ccm.user_name
> FROM serviceconfig sc JOIN serviceconfigmapping m ON sc.service_config_id = 
> m.service_config_id
> JOIN clusterconfig c ON m.config_id = c.config_id
> JOIN clusterconfigmapping ccm ON c.type_name = ccm.type_name AND 
> c.version_tag = ccm.version_tag
> WHERE sc.service_name = 'SPARK2' and c.type_name = 
> 'spark2-javaopts-properties'
> ORDER BY sc.version, c.type_name;
>  service_name | version |   user_name|  to_timestamp  | stack_id 
> |   note| type_version | version_tag | 
> type_name  | stack_id | clusterconfig_created  | 
> clusterconfigmapping_created | selected | user_name
> --+-+++--+---+--+-++--++--+--+---
>  SPARK2   |   1 | admin  | 2017-07-11 20:17:01+00 |3 
> | Initial configurations for Spark2 |1 | version1| 
> spark2-javaopts-properties |3 | 2017-07-11 20:17:00+00 | 2017-07-11 
> 20:17:01+00   |0 | admin
>  SPARK2   |   2 | ambari-upgrade | 2017-07-13 20:33:50+00 |3 
> |   |1 | version1| 
> spark2-javaopts-properties |3 | 2017-07-11 20:17:00+00 | 2017-07-11 
> 20:17:01+00   |0 | admin
> (2 rows)
> ```
> 
> This is because the config type has a single config with an empty value, so 
> perhaps UI never selects it.
> ```
> 
>   
> content
> Spark2-javaopts-properties
>  
> 
>   
> 
> ```
> 
> Fix will be for Ambari 2.5.2 upgrade to select exactly once config for each 
> config type if none are selected.
> The only config type to hit this issue is spark2-javaopts-properties
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog252.java
>  6d2ab84 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/SPARK/configuration/spark-javaopts-properties.xml
>  a197e34 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SPARK2/configuration/spark2-javaopts-properties.xml
>  f8d50fc 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SPARK2/metainfo.xml
>  bf75f47 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2/services/SPARK/configuration/spark-javaopts-properties.xml
>  77a7282 
> 
> 
> Diff: https://reviews.apache.org/r/60886/diff/2/
> 
> 
> Testing
> ---
> 
> Verified on local cluster. Unit test for UpgradeCatalog252Test.java passed.
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Re: Review Request 60914: DB consistency checker throws errors for missing 'product-info' configs after Ambari upgrade

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60914/#review180684
---


Ship it!




Ship It!

- Jonathan Hurley


On July 17, 2017, 11:49 a.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60914/
> ---
> 
> (Updated July 17, 2017, 11:49 a.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley and Nate Cole.
> 
> 
> Bugs: AMBARI-21498
> https://issues.apache.org/jira/browse/AMBARI-21498
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> DB consistency checker throws errors for missing 'product-info' configs after 
> Ambari upgrade
> AMBARI-21364 fixed the missing 'parquet-logging' but 'product-info' is still 
> missing for Smartsense
> 
> STR
> Deployed cluster with Ambari version: 2.5.1.0-159 and HDP version: 2.6.1.0-129
> Upgrade Ambari to 2.5.2.0-105
> Run "ambari-server start"
> 
> 
> Live openstack cluster : 172.22.124.150 (Ambari Server)
> ambari-server-check-database.log
> {code}
> 2017-07-07 18:09:05,678 ERROR - Required config(s): product-info is(are) not 
> available for service SMARTSENSE with service config version 1 in cluster cl1
> {code}
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog252.java
>  6d2ab843ac 
> 
> 
> Diff: https://reviews.apache.org/r/60914/diff/1/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Review Request 60914: DB consistency checker throws errors for missing 'product-info' configs after Ambari upgrade

[Dmitro Lisnichenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60914/
---

Review request for Ambari, Jonathan Hurley and Nate Cole.


Bugs: AMBARI-21498
https://issues.apache.org/jira/browse/AMBARI-21498


Repository: ambari


Description
---

DB consistency checker throws errors for missing 'product-info' configs after 
Ambari upgrade
AMBARI-21364 fixed the missing 'parquet-logging' but 'product-info' is still 
missing for Smartsense

STR
Deployed cluster with Ambari version: 2.5.1.0-159 and HDP version: 2.6.1.0-129
Upgrade Ambari to 2.5.2.0-105
Run "ambari-server start"


Live openstack cluster : 172.22.124.150 (Ambari Server)
ambari-server-check-database.log
{code}
2017-07-07 18:09:05,678 ERROR - Required config(s): product-info is(are) not 
available for service SMARTSENSE with service config version 1 in cluster cl1
{code}


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog252.java
 6d2ab843ac 


Diff: https://reviews.apache.org/r/60914/diff/1/


Testing
---

mvn clean test


Thanks,

Dmitro Lisnichenko

Re: Review Request 60906: Add SOLR to BigInsights 4.2.5 stack

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60906/#review180682
---


Ship it!




Ship It!

- Jonathan Hurley


On July 17, 2017, 8:08 a.m., Oliver Szabo wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60906/
> ---
> 
> (Updated July 17, 2017, 8:08 a.m.)
> 
> 
> Review request for Ambari, Attila Doroszlai, Jonathan Hurley, Robert 
> Nettleton, Sumit Mohanty, Sebastian Toader, and Sid Wagle.
> 
> 
> Bugs: AMBARI-21487
> https://issues.apache.org/jira/browse/AMBARI-21487
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> SOLR is missing from 4.2.5 stack. I moved the solr stack code from 
> common-services to stacks/BigInsights/4.2.5/
> 
> note: in the stack metainfo.xml there is an  tag for 4.2 stack, that 
> means a solr-site config type is added during ambari upgrade (because of 
> on-ambari-upgrade add="true" tags), because of that solr needs to be 
> restarted. if i change the on-ambari-upgrade add flag to false, its not 
> needed to restart SOLR, but because of the  of metainfo.xml, db 
> check will fail. So i kept that behaviour.
> 
> next: HDP upgrade
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/alerts.json
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/configuration/ranger-solr-audit.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/configuration/ranger-solr-plugin-properties.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/configuration/ranger-solr-policymgr-ssl.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/configuration/ranger-solr-security.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/configuration/solr-client-log4j.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/configuration/solr-env.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/configuration/solr-log4j.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/configuration/solr-xml.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/kerberos.json
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/metainfo.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/package/scripts/params.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/package/scripts/service_check.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/package/scripts/setup_ranger_solr.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/package/scripts/setup_solr.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/package/scripts/solr.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/package/scripts/solr_client.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/package/scripts/status_params.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/package/templates/solr_jaas.conf.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/properties/solr-client-log4j.properties.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/properties/solr-env.sh.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/properties/solr-log4j.properties.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/properties/solr.xml.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/quicklinks/quicklinks.json
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/themes/theme.json
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/60906/diff/1/
> 
> 
> Testing
> ---
> 
> test manually:
> Upgrade ambari -> start/stop solr.
> 
> 
> Thanks,
> 
> Oliver Szabo
> 
>

Re: Review Request 60886: AMBARI-21481. Upgrading IOP cluster with Spark2 to Ambari 2.5.2 fails on start because config mapping spark2-javaopts-properties is never selected

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60886/#review180681
---



So, this is an env file with no content? Doesn't that mean that it's optional 
and doesn't need to be created? Can we simply use the  tag (see 
AMBARI-21364) and have the DB checker skip it?

- Jonathan Hurley


On July 14, 2017, 8:20 p.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60886/
> ---
> 
> (Updated July 14, 2017, 8:20 p.m.)
> 
> 
> Review request for Ambari, Andrii Tkach, Dmytro Grinenko, Jonathan Hurley, 
> Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-21481
> https://issues.apache.org/jira/browse/AMBARI-21481
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> STR:
> 
> * Install BigInsights 4.2.5 with Spark2
> * Upgrade to Ambari 2.5.2
> * Start Ambari, which will fail in the DB consistency check
> 
> ```
> 2017-07-07 01:07:01,881 ERROR - You have non selected configs: 
> spark2-javaopts-properties for service SPARK2 from cluster c1!
> ```
> 
> Looks like this config is never selected,
> 
> ```
> ambari=> SELECT sc.service_name, sc.version, sc.user_name, 
> TO_TIMESTAMP(sc.create_timestamp/1000), sc.stack_id, sc.note,
> c.version AS type_version, c.version_tag, c.type_name, c.stack_id, 
> TO_TIMESTAMP(c.create_timestamp/1000) AS clusterconfig_created,
> TO_TIMESTAMP(ccm.create_timestamp/1000) AS clusterconfigmapping_created, 
> ccm.selected, ccm.user_name
> FROM serviceconfig sc JOIN serviceconfigmapping m ON sc.service_config_id = 
> m.service_config_id
> JOIN clusterconfig c ON m.config_id = c.config_id
> JOIN clusterconfigmapping ccm ON c.type_name = ccm.type_name AND 
> c.version_tag = ccm.version_tag
> WHERE sc.service_name = 'SPARK2' and c.type_name = 
> 'spark2-javaopts-properties'
> ORDER BY sc.version, c.type_name;
>  service_name | version |   user_name|  to_timestamp  | stack_id 
> |   note| type_version | version_tag | 
> type_name  | stack_id | clusterconfig_created  | 
> clusterconfigmapping_created | selected | user_name
> --+-+++--+---+--+-++--++--+--+---
>  SPARK2   |   1 | admin  | 2017-07-11 20:17:01+00 |3 
> | Initial configurations for Spark2 |1 | version1| 
> spark2-javaopts-properties |3 | 2017-07-11 20:17:00+00 | 2017-07-11 
> 20:17:01+00   |0 | admin
>  SPARK2   |   2 | ambari-upgrade | 2017-07-13 20:33:50+00 |3 
> |   |1 | version1| 
> spark2-javaopts-properties |3 | 2017-07-11 20:17:00+00 | 2017-07-11 
> 20:17:01+00   |0 | admin
> (2 rows)
> ```
> 
> This is because the config type has a single config with an empty value, so 
> perhaps UI never selects it.
> ```
> 
>   
> content
> Spark2-javaopts-properties
>  
> 
>   
> 
> ```
> 
> Fix will be for Ambari 2.5.2 upgrade to select exactly once config for each 
> config type if none are selected.
> The only config type to hit this issue is spark2-javaopts-properties
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog252.java
>  6d2ab84 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/SPARK/configuration/spark-javaopts-properties.xml
>  a197e34 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SPARK2/configuration/spark2-javaopts-properties.xml
>  f8d50fc 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SPARK2/metainfo.xml
>  bf75f47 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2/services/SPARK/configuration/spark-javaopts-properties.xml
>  77a7282 
> 
> 
> Diff: https://reviews.apache.org/r/60886/diff/2/
> 
> 
> Testing
> ---
> 
> Verified on local cluster. Unit test for UpgradeCatalog252Test.java passed.
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Review Request 60912: Fix HDP base services to handle changed config paths

[Andrew Onischuk]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60912/
---

Review request for Ambari, Alejandro Fernandez, Myroslav Papirkovskyy, and Sid 
Wagle.


Bugs: AMBARI-21496
https://issues.apache.org/jira/browse/AMBARI-21496


Repository: ambari


Description
---


Diffs
-

  
ambari-server/src/main/resources/common-services/ACCUMULO/1.6.1.2.2.0/package/scripts/accumulo_configuration.py
 f65b68a 
  
ambari-server/src/main/resources/common-services/ACCUMULO/1.6.1.2.2.0/package/scripts/params.py
 53cf002 
  
ambari-server/src/main/resources/common-services/ACCUMULO/1.6.1.2.2.0/package/scripts/status_params.py
 9fc89f1 
  
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/params.py
 acf420e 
  
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/ams.py
 a929847 
  
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/hbase.py
 e20ff1d 
  
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py
 919f26d 
  
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/status_params.py
 6057a8b 
  
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata.py
 36c4598 
  
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py
 d26df33 
  
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/setup_ranger_atlas.py
 c47c75c 
  
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/status_params.py
 852a9cb 
  
ambari-server/src/main/resources/common-services/DRUID/0.9.2/package/scripts/params.py
 b1d9472 
  
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/params.py
 2942992 
  
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/params_linux.py
 3773918 
  
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/status_params.py
 0a0e370 
  
ambari-server/src/main/resources/common-services/FLUME/1.4.0.2.0/package/scripts/params.py
 b143941 
  
ambari-server/src/main/resources/common-services/FLUME/1.4.0.2.0/package/scripts/params_linux.py
 53b6be0 
  
ambari-server/src/main/resources/common-services/GANGLIA/3.5.0/package/scripts/params.py
 8473013 
  
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/params.py
 e2dbd7b 
  
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/hbase.py
 8ad802e 
  
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py
 3177643 
  
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/setup_ranger_hbase.py
 d32dce1 
  
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/status_params.py
 bc75c78 
  
ambari-server/src/main/resources/common-services/HBASE/2.0.0.3.0/package/scripts/hbase.py
 fced4fc 
  
ambari-server/src/main/resources/common-services/HBASE/2.0.0.3.0/package/scripts/params_linux.py
 b7e2b89 
  
ambari-server/src/main/resources/common-services/HBASE/2.0.0.3.0/package/scripts/setup_ranger_hbase.py
 d32dce1 
  
ambari-server/src/main/resources/common-services/HBASE/2.0.0.3.0/package/scripts/status_params.py
 bc75c78 
  
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs.py
 d9b62e2 
  
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/install_params.py
 fe488c3 
  
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py
 e88dbdd 
  
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_windows.py
 1e47c29 
  
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/setup_ranger_hdfs.py
 47c6e35 
  
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/status_params.py
 153f9a6 
  
ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/hdfs.py
 d9b62e2 
  
ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/install_params.py
 fe488c3 
  
ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/params_linux.py
 58bb65f 
  
ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/params_windows.py
 70d95a6 
  
ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/setup_ranger_hdfs.py
 d0d92d7 
  
ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/status_params.py
 153f9a6 
  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hcat.py
 faa8c7e 
  

Re: Review Request 60910: Fix PERF stack scripts to handle changed config paths

[Victor Galgo via Review Board]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60910/#review180676
---


Ship it!




Ship It!

- Victor Galgo


On July 17, 2017, 1:34 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60910/
> ---
> 
> (Updated July 17, 2017, 1:34 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Myroslav Papirkovskyy, and 
> Sid Wagle.
> 
> 
> Bugs: AMBARI-21493
> https://issues.apache.org/jira/browse/AMBARI-21493
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/get_not_managed_resources.py
>  4af636b 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py
>  2b3df5f 
>   
> ambari-common/src/main/python/resource_management/libraries/providers/xml_config.py
>  28697bf 
>   ambari-common/src/main/python/resource_management/libraries/script/dummy.py 
> ad5f2a6 
>   
> ambari-common/src/main/python/resource_management/libraries/script/script.py 
> a08feab 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/params.py
>  9abd2fe 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/shared_initialization.py
>  36a202f 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-ANY/scripts/params.py
>  4052d1d 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/params.py
>  6193c11 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/repo_initialization.py
>  a35dce7 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/params.py
>  70ebfeb 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/shared_initialization.py
>  148d235 
>   
> ambari-server/src/main/resources/stacks/HDP/3.0/hooks/after-INSTALL/scripts/params.py
>  5dcd39b 
>   
> ambari-server/src/main/resources/stacks/HDP/3.0/hooks/after-INSTALL/scripts/shared_initialization.py
>  e9f2283 
>   
> ambari-server/src/main/resources/stacks/HDP/3.0/hooks/before-ANY/scripts/params.py
>  9be9101 
>   
> ambari-server/src/main/resources/stacks/HDP/3.0/hooks/before-INSTALL/scripts/params.py
>  6193c11 
>   
> ambari-server/src/main/resources/stacks/HDP/3.0/hooks/before-START/scripts/params.py
>  5a5361c 
>   
> ambari-server/src/main/resources/stacks/PERF/1.0/hooks/before-ANY/scripts/params.py
>  2c2c901 
>   
> ambari-server/src/main/resources/stacks/PERF/1.0/services/FAKEHBASE/metainfo.xml
>  66d5a29 
>   
> ambari-server/src/main/resources/stacks/PERF/1.0/services/FAKEHDFS/metainfo.xml
>  13b10e0 
>   
> ambari-server/src/main/resources/stacks/PERF/1.0/services/FAKEHDFS/package/scripts/params.py
>  8068441 
>   
> ambari-server/src/main/resources/stacks/PERF/1.0/services/KERBEROS/package/scripts/params.py
>  4eb5b02 
> 
> 
> Diff: https://reviews.apache.org/r/60910/diff/1/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 60911: BI 4.2 Ranger Needs Properties Added on Ambari Upgrade

[Attila Doroszlai]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60911/#review180675
---


Ship it!




Ship It!

- Attila Doroszlai


On July 17, 2017, 3:40 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60911/
> ---
> 
> (Updated July 17, 2017, 3:40 p.m.)
> 
> 
> Review request for Ambari, Attila Doroszlai, Alejandro Fernandez, and Sid 
> Wagle.
> 
> 
> Bugs: AMBARI-21494
> https://issues.apache.org/jira/browse/AMBARI-21494
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> The following two properties were added to Ranger 0.5.0 for Ambari 2.5.0. 
> However, since BigInsight 4.2.0 doesn't extend common services, these need to 
> be added for the upgrade from Ambari 2.4.2 to 2.5:
> 
> - ranger.ldap.binddn.credential.alias
> - ranger.ldap.ad.binddn.credential.alias
> 
> Reference commit: 
> https://github.com/apache/ambari/commit/0d9d31fe3f8ace4a7ad710df34fe5ccba4b66df1#diff-1ae96dddc715cc510335ca0bcb698c23
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2/services/RANGER/configuration/ranger-admin-site.xml
>  dcc652d1c8 
> 
> 
> Diff: https://reviews.apache.org/r/60911/diff/1/
> 
> 
> Testing
> ---
> 
> Manual Ambari Upgrade
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Review Request 60911: BI 4.2 Ranger Needs Properties Added on Ambari Upgrade

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60911/
---

Review request for Ambari, Attila Doroszlai, Alejandro Fernandez, and Sid Wagle.


Bugs: AMBARI-21494
https://issues.apache.org/jira/browse/AMBARI-21494


Repository: ambari


Description
---

The following two properties were added to Ranger 0.5.0 for Ambari 2.5.0. 
However, since BigInsight 4.2.0 doesn't extend common services, these need to 
be added for the upgrade from Ambari 2.4.2 to 2.5:

- ranger.ldap.binddn.credential.alias
- ranger.ldap.ad.binddn.credential.alias

Reference commit: 
https://github.com/apache/ambari/commit/0d9d31fe3f8ace4a7ad710df34fe5ccba4b66df1#diff-1ae96dddc715cc510335ca0bcb698c23


Diffs
-

  
ambari-server/src/main/resources/stacks/BigInsights/4.2/services/RANGER/configuration/ranger-admin-site.xml
 dcc652d1c8 


Diff: https://reviews.apache.org/r/60911/diff/1/


Testing
---

Manual Ambari Upgrade


Thanks,

Jonathan Hurley

Review Request 60910: Fix PERF stack scripts to handle changed config paths

[Andrew Onischuk]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60910/
---

Review request for Ambari, Alejandro Fernandez, Myroslav Papirkovskyy, and Sid 
Wagle.


Bugs: AMBARI-21493
https://issues.apache.org/jira/browse/AMBARI-21493


Repository: ambari


Description
---


Diffs
-

  
ambari-common/src/main/python/resource_management/libraries/functions/get_not_managed_resources.py
 4af636b 
  
ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py
 2b3df5f 
  
ambari-common/src/main/python/resource_management/libraries/providers/xml_config.py
 28697bf 
  ambari-common/src/main/python/resource_management/libraries/script/dummy.py 
ad5f2a6 
  ambari-common/src/main/python/resource_management/libraries/script/script.py 
a08feab 
  
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/params.py
 9abd2fe 
  
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/shared_initialization.py
 36a202f 
  
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-ANY/scripts/params.py
 4052d1d 
  
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/params.py
 6193c11 
  
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/repo_initialization.py
 a35dce7 
  
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/params.py
 70ebfeb 
  
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/shared_initialization.py
 148d235 
  
ambari-server/src/main/resources/stacks/HDP/3.0/hooks/after-INSTALL/scripts/params.py
 5dcd39b 
  
ambari-server/src/main/resources/stacks/HDP/3.0/hooks/after-INSTALL/scripts/shared_initialization.py
 e9f2283 
  
ambari-server/src/main/resources/stacks/HDP/3.0/hooks/before-ANY/scripts/params.py
 9be9101 
  
ambari-server/src/main/resources/stacks/HDP/3.0/hooks/before-INSTALL/scripts/params.py
 6193c11 
  
ambari-server/src/main/resources/stacks/HDP/3.0/hooks/before-START/scripts/params.py
 5a5361c 
  
ambari-server/src/main/resources/stacks/PERF/1.0/hooks/before-ANY/scripts/params.py
 2c2c901 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/FAKEHBASE/metainfo.xml
 66d5a29 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/FAKEHDFS/metainfo.xml 
13b10e0 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/FAKEHDFS/package/scripts/params.py
 8068441 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/KERBEROS/package/scripts/params.py
 4eb5b02 


Diff: https://reviews.apache.org/r/60910/diff/1/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Re: Review Request 60859: Zeppelin does not start and returns: params.zookeeper_znode_parent not in interpreter['properties']['phoenix.url']: KeyError: 'phoenix.url'

[Gaurav Nagar]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60859/#review180665
---


Ship it!




Ship It!

- Gaurav Nagar


On July 14, 2017, 11:18 a.m., Prabhjyot Singh wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60859/
> ---
> 
> (Updated July 14, 2017, 11:18 a.m.)
> 
> 
> Review request for Ambari, Attila Doroszlai, Alejandro Fernandez, DIPAYAN 
> BHOWMICK, Jayush Luniya, Rohit Choudhary, Renjith Kamath, Sumit Mohanty, and 
> venkat sairam.
> 
> 
> Bugs: AMBARI-21473
> https://issues.apache.org/jira/browse/AMBARI-21473
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Zeppelin does not start and returns: params.zookeeper_znode_parent not in 
> interpreter['properties']['phoenix.url']: KeyError: 'phoenix.url'
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/master.py
>  0013ab0eb8 
> 
> 
> Diff: https://reviews.apache.org/r/60859/diff/1/
> 
> 
> Testing
> ---
> 
> Manually on CentOS 6 with kerberos enabled
> 
> 
> Thanks,
> 
> Prabhjyot Singh
> 
>

Re: Review Request 60859: Zeppelin does not start and returns: params.zookeeper_znode_parent not in interpreter['properties']['phoenix.url']: KeyError: 'phoenix.url'

[Nitiraj Rathore]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60859/#review180663
---


Ship it!




Ship It!

- Nitiraj Rathore


On July 14, 2017, 11:18 a.m., Prabhjyot Singh wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60859/
> ---
> 
> (Updated July 14, 2017, 11:18 a.m.)
> 
> 
> Review request for Ambari, Attila Doroszlai, Alejandro Fernandez, DIPAYAN 
> BHOWMICK, Jayush Luniya, Rohit Choudhary, Renjith Kamath, Sumit Mohanty, and 
> venkat sairam.
> 
> 
> Bugs: AMBARI-21473
> https://issues.apache.org/jira/browse/AMBARI-21473
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Zeppelin does not start and returns: params.zookeeper_znode_parent not in 
> interpreter['properties']['phoenix.url']: KeyError: 'phoenix.url'
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/master.py
>  0013ab0eb8 
> 
> 
> Diff: https://reviews.apache.org/r/60859/diff/1/
> 
> 
> Testing
> ---
> 
> Manually on CentOS 6 with kerberos enabled
> 
> 
> Thanks,
> 
> Prabhjyot Singh
> 
>

Re: Review Request 60859: Zeppelin does not start and returns: params.zookeeper_znode_parent not in interpreter['properties']['phoenix.url']: KeyError: 'phoenix.url'

[venkat sairam]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60859/#review180664
---


Ship it!




Ship It!

- venkat sairam


On July 14, 2017, 11:18 a.m., Prabhjyot Singh wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60859/
> ---
> 
> (Updated July 14, 2017, 11:18 a.m.)
> 
> 
> Review request for Ambari, Attila Doroszlai, Alejandro Fernandez, DIPAYAN 
> BHOWMICK, Jayush Luniya, Rohit Choudhary, Renjith Kamath, Sumit Mohanty, and 
> venkat sairam.
> 
> 
> Bugs: AMBARI-21473
> https://issues.apache.org/jira/browse/AMBARI-21473
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Zeppelin does not start and returns: params.zookeeper_znode_parent not in 
> interpreter['properties']['phoenix.url']: KeyError: 'phoenix.url'
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/master.py
>  0013ab0eb8 
> 
> 
> Diff: https://reviews.apache.org/r/60859/diff/1/
> 
> 
> Testing
> ---
> 
> Manually on CentOS 6 with kerberos enabled
> 
> 
> Thanks,
> 
> Prabhjyot Singh
> 
>