subject:"Re\: Review Request 60886\: AMBARI\-21481. Upgrading IOP cluster with Spark2 to Ambari 2.5.2 fails on start because config mapping spark2\-javaopts\-properties is never selected"

Re: Review Request 60886: AMBARI-21481. Upgrading IOP cluster with Spark2 to Ambari 2.5.2 fails on start because config mapping spark2-javaopts-properties is never selected

2017-07-17 Thread Alejandro Fernandez


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60886/
---

(Updated July 17, 2017, 9:26 p.m.)


Review request for Ambari, Andrii Tkach, Dmytro Grinenko, Jonathan Hurley, 
Sumit Mohanty, and Sid Wagle.


Bugs: AMBARI-21481
https://issues.apache.org/jira/browse/AMBARI-21481


Repository: ambari


Description
---

STR:

* Install BigInsights 4.2.5 with Spark2
* Upgrade to Ambari 2.5.2
* Start Ambari, which will fail in the DB consistency check

```
2017-07-07 01:07:01,881 ERROR - You have non selected configs: 
spark2-javaopts-properties for service SPARK2 from cluster c1!
```

Looks like this config is never selected,

```
ambari=> SELECT sc.service_name, sc.version, sc.user_name, 
TO_TIMESTAMP(sc.create_timestamp/1000), sc.stack_id, sc.note,
c.version AS type_version, c.version_tag, c.type_name, c.stack_id, 
TO_TIMESTAMP(c.create_timestamp/1000) AS clusterconfig_created,
TO_TIMESTAMP(ccm.create_timestamp/1000) AS clusterconfigmapping_created, 
ccm.selected, ccm.user_name
FROM serviceconfig sc JOIN serviceconfigmapping m ON sc.service_config_id = 
m.service_config_id
JOIN clusterconfig c ON m.config_id = c.config_id
JOIN clusterconfigmapping ccm ON c.type_name = ccm.type_name AND c.version_tag 
= ccm.version_tag
WHERE sc.service_name = 'SPARK2' and c.type_name = 'spark2-javaopts-properties'
ORDER BY sc.version, c.type_name;
 service_name | version |   user_name|  to_timestamp  | stack_id |  
 note| type_version | version_tag | 
type_name  | stack_id | clusterconfig_created  | 
clusterconfigmapping_created | selected | user_name
--+-+++--+---+--+-++--++--+--+---
 SPARK2   |   1 | admin  | 2017-07-11 20:17:01+00 |3 | 
Initial configurations for Spark2 |1 | version1| 
spark2-javaopts-properties |3 | 2017-07-11 20:17:00+00 | 2017-07-11 
20:17:01+00   |0 | admin
 SPARK2   |   2 | ambari-upgrade | 2017-07-13 20:33:50+00 |3 |  
 |1 | version1| 
spark2-javaopts-properties |3 | 2017-07-11 20:17:00+00 | 2017-07-11 
20:17:01+00   |0 | admin
(2 rows)
```

This is because the config type has a single config with an empty value, so 
perhaps UI never selects it.
```

  
content
Spark2-javaopts-properties
 

  

```

Fix will be for Ambari 2.5.2 upgrade to select exactly once config for each 
config type if none are selected.
The only config type to hit this issue is spark2-javaopts-properties


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog252.java
 3c8686c 
  
ambari-server/src/main/resources/stacks/BigInsights/4.0/services/SPARK/configuration/spark-javaopts-properties.xml
 a197e34 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SPARK2/configuration/spark2-javaopts-properties.xml
 f8d50fc 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SPARK2/metainfo.xml
 bf75f47 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2/services/SPARK/configuration/spark-javaopts-properties.xml
 77a7282 


Diff: https://reviews.apache.org/r/60886/diff/5/

Changes: https://reviews.apache.org/r/60886/diff/4-5/


Testing
---

Verified on local cluster. Unit test for UpgradeCatalog252Test.java passed.


Thanks,

Alejandro Fernandez

Re: Review Request 60886: AMBARI-21481. Upgrading IOP cluster with Spark2 to Ambari 2.5.2 fails on start because config mapping spark2-javaopts-properties is never selected

2017-07-17 Thread Jonathan Hurley


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60886/#review180724
---


Fix it, then Ship it!





ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog252.java
Lines 218-224 (patched)


Can we change this so that it doesn't mention stack upgrades or HDP/BI? The 
problem is on Ambari upgrade, right? And it could happen for any config?



ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog252.java
Lines 228 (patched)


Can we do away with this check? I don't think we'd be upgrading without any 
clusters :)



ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog252.java
Lines 230 (patched)


This should probably log the types we're enforcing now, which seems to be 
only 1.


- Jonathan Hurley


On July 17, 2017, 3:15 p.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60886/
> ---
> 
> (Updated July 17, 2017, 3:15 p.m.)
> 
> 
> Review request for Ambari, Andrii Tkach, Dmytro Grinenko, Jonathan Hurley, 
> Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-21481
> https://issues.apache.org/jira/browse/AMBARI-21481
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> STR:
> 
> * Install BigInsights 4.2.5 with Spark2
> * Upgrade to Ambari 2.5.2
> * Start Ambari, which will fail in the DB consistency check
> 
> ```
> 2017-07-07 01:07:01,881 ERROR - You have non selected configs: 
> spark2-javaopts-properties for service SPARK2 from cluster c1!
> ```
> 
> Looks like this config is never selected,
> 
> ```
> ambari=> SELECT sc.service_name, sc.version, sc.user_name, 
> TO_TIMESTAMP(sc.create_timestamp/1000), sc.stack_id, sc.note,
> c.version AS type_version, c.version_tag, c.type_name, c.stack_id, 
> TO_TIMESTAMP(c.create_timestamp/1000) AS clusterconfig_created,
> TO_TIMESTAMP(ccm.create_timestamp/1000) AS clusterconfigmapping_created, 
> ccm.selected, ccm.user_name
> FROM serviceconfig sc JOIN serviceconfigmapping m ON sc.service_config_id = 
> m.service_config_id
> JOIN clusterconfig c ON m.config_id = c.config_id
> JOIN clusterconfigmapping ccm ON c.type_name = ccm.type_name AND 
> c.version_tag = ccm.version_tag
> WHERE sc.service_name = 'SPARK2' and c.type_name = 
> 'spark2-javaopts-properties'
> ORDER BY sc.version, c.type_name;
>  service_name | version |   user_name|  to_timestamp  | stack_id 
> |   note| type_version | version_tag | 
> type_name  | stack_id | clusterconfig_created  | 
> clusterconfigmapping_created | selected | user_name
> --+-+++--+---+--+-++--++--+--+---
>  SPARK2   |   1 | admin  | 2017-07-11 20:17:01+00 |3 
> | Initial configurations for Spark2 |1 | version1| 
> spark2-javaopts-properties |3 | 2017-07-11 20:17:00+00 | 2017-07-11 
> 20:17:01+00   |0 | admin
>  SPARK2   |   2 | ambari-upgrade | 2017-07-13 20:33:50+00 |3 
> |   |1 | version1| 
> spark2-javaopts-properties |3 | 2017-07-11 20:17:00+00 | 2017-07-11 
> 20:17:01+00   |0 | admin
> (2 rows)
> ```
> 
> This is because the config type has a single config with an empty value, so 
> perhaps UI never selects it.
> ```
> 
>   
> content
> Spark2-javaopts-properties
>  
> 
>   
> 
> ```
> 
> Fix will be for Ambari 2.5.2 upgrade to select exactly once config for each 
> config type if none are selected.
> The only config type to hit this issue is spark2-javaopts-properties
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog252.java
>  3c8686c 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/SPARK/configuration/spark-javaopts-properties.xml
>  a197e34 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SPARK2/configuration/spark2-javaopts-properties.xml
>  f8d50fc 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SPARK2/metainfo.xml
>  bf75f47 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2/services/SPARK/configuration/spark-javaopts-properties.xml
>  77a7282 
> 
> 
> Diff: https://reviews.apache.org/r/60886/diff/4/
> 
> 
> Testing
> ---
> 
> Verified on local

Re: Review Request 60886: AMBARI-21481. Upgrading IOP cluster with Spark2 to Ambari 2.5.2 fails on start because config mapping spark2-javaopts-properties is never selected

2017-07-17 Thread Alejandro Fernandez



> On July 17, 2017, 6:26 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SPARK2/metainfo.xml
> > Line 82 (original), 82 (patched)
> > 
> >
> > Why indented?

Fixed


- Alejandro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60886/#review180710
---


On July 17, 2017, 7:15 p.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60886/
> ---
> 
> (Updated July 17, 2017, 7:15 p.m.)
> 
> 
> Review request for Ambari, Andrii Tkach, Dmytro Grinenko, Jonathan Hurley, 
> Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-21481
> https://issues.apache.org/jira/browse/AMBARI-21481
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> STR:
> 
> * Install BigInsights 4.2.5 with Spark2
> * Upgrade to Ambari 2.5.2
> * Start Ambari, which will fail in the DB consistency check
> 
> ```
> 2017-07-07 01:07:01,881 ERROR - You have non selected configs: 
> spark2-javaopts-properties for service SPARK2 from cluster c1!
> ```
> 
> Looks like this config is never selected,
> 
> ```
> ambari=> SELECT sc.service_name, sc.version, sc.user_name, 
> TO_TIMESTAMP(sc.create_timestamp/1000), sc.stack_id, sc.note,
> c.version AS type_version, c.version_tag, c.type_name, c.stack_id, 
> TO_TIMESTAMP(c.create_timestamp/1000) AS clusterconfig_created,
> TO_TIMESTAMP(ccm.create_timestamp/1000) AS clusterconfigmapping_created, 
> ccm.selected, ccm.user_name
> FROM serviceconfig sc JOIN serviceconfigmapping m ON sc.service_config_id = 
> m.service_config_id
> JOIN clusterconfig c ON m.config_id = c.config_id
> JOIN clusterconfigmapping ccm ON c.type_name = ccm.type_name AND 
> c.version_tag = ccm.version_tag
> WHERE sc.service_name = 'SPARK2' and c.type_name = 
> 'spark2-javaopts-properties'
> ORDER BY sc.version, c.type_name;
>  service_name | version |   user_name|  to_timestamp  | stack_id 
> |   note| type_version | version_tag | 
> type_name  | stack_id | clusterconfig_created  | 
> clusterconfigmapping_created | selected | user_name
> --+-+++--+---+--+-++--++--+--+---
>  SPARK2   |   1 | admin  | 2017-07-11 20:17:01+00 |3 
> | Initial configurations for Spark2 |1 | version1| 
> spark2-javaopts-properties |3 | 2017-07-11 20:17:00+00 | 2017-07-11 
> 20:17:01+00   |0 | admin
>  SPARK2   |   2 | ambari-upgrade | 2017-07-13 20:33:50+00 |3 
> |   |1 | version1| 
> spark2-javaopts-properties |3 | 2017-07-11 20:17:00+00 | 2017-07-11 
> 20:17:01+00   |0 | admin
> (2 rows)
> ```
> 
> This is because the config type has a single config with an empty value, so 
> perhaps UI never selects it.
> ```
> 
>   
> content
> Spark2-javaopts-properties
>  
> 
>   
> 
> ```
> 
> Fix will be for Ambari 2.5.2 upgrade to select exactly once config for each 
> config type if none are selected.
> The only config type to hit this issue is spark2-javaopts-properties
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog252.java
>  3c8686c 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/SPARK/configuration/spark-javaopts-properties.xml
>  a197e34 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SPARK2/configuration/spark2-javaopts-properties.xml
>  f8d50fc 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SPARK2/metainfo.xml
>  bf75f47 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2/services/SPARK/configuration/spark-javaopts-properties.xml
>  77a7282 
> 
> 
> Diff: https://reviews.apache.org/r/60886/diff/4/
> 
> 
> Testing
> ---
> 
> Verified on local cluster. Unit test for UpgradeCatalog252Test.java passed.
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Re: Review Request 60886: AMBARI-21481. Upgrading IOP cluster with Spark2 to Ambari 2.5.2 fails on start because config mapping spark2-javaopts-properties is never selected

2017-07-17 Thread Alejandro Fernandez


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60886/
---

(Updated July 17, 2017, 7:15 p.m.)


Review request for Ambari, Andrii Tkach, Dmytro Grinenko, Jonathan Hurley, 
Sumit Mohanty, and Sid Wagle.


Bugs: AMBARI-21481
https://issues.apache.org/jira/browse/AMBARI-21481


Repository: ambari


Description
---

STR:

* Install BigInsights 4.2.5 with Spark2
* Upgrade to Ambari 2.5.2
* Start Ambari, which will fail in the DB consistency check

```
2017-07-07 01:07:01,881 ERROR - You have non selected configs: 
spark2-javaopts-properties for service SPARK2 from cluster c1!
```

Looks like this config is never selected,

```
ambari=> SELECT sc.service_name, sc.version, sc.user_name, 
TO_TIMESTAMP(sc.create_timestamp/1000), sc.stack_id, sc.note,
c.version AS type_version, c.version_tag, c.type_name, c.stack_id, 
TO_TIMESTAMP(c.create_timestamp/1000) AS clusterconfig_created,
TO_TIMESTAMP(ccm.create_timestamp/1000) AS clusterconfigmapping_created, 
ccm.selected, ccm.user_name
FROM serviceconfig sc JOIN serviceconfigmapping m ON sc.service_config_id = 
m.service_config_id
JOIN clusterconfig c ON m.config_id = c.config_id
JOIN clusterconfigmapping ccm ON c.type_name = ccm.type_name AND c.version_tag 
= ccm.version_tag
WHERE sc.service_name = 'SPARK2' and c.type_name = 'spark2-javaopts-properties'
ORDER BY sc.version, c.type_name;
 service_name | version |   user_name|  to_timestamp  | stack_id |  
 note| type_version | version_tag | 
type_name  | stack_id | clusterconfig_created  | 
clusterconfigmapping_created | selected | user_name
--+-+++--+---+--+-++--++--+--+---
 SPARK2   |   1 | admin  | 2017-07-11 20:17:01+00 |3 | 
Initial configurations for Spark2 |1 | version1| 
spark2-javaopts-properties |3 | 2017-07-11 20:17:00+00 | 2017-07-11 
20:17:01+00   |0 | admin
 SPARK2   |   2 | ambari-upgrade | 2017-07-13 20:33:50+00 |3 |  
 |1 | version1| 
spark2-javaopts-properties |3 | 2017-07-11 20:17:00+00 | 2017-07-11 
20:17:01+00   |0 | admin
(2 rows)
```

This is because the config type has a single config with an empty value, so 
perhaps UI never selects it.
```

  
content
Spark2-javaopts-properties
 

  

```

Fix will be for Ambari 2.5.2 upgrade to select exactly once config for each 
config type if none are selected.
The only config type to hit this issue is spark2-javaopts-properties


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog252.java
 3c8686c 
  
ambari-server/src/main/resources/stacks/BigInsights/4.0/services/SPARK/configuration/spark-javaopts-properties.xml
 a197e34 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SPARK2/configuration/spark2-javaopts-properties.xml
 f8d50fc 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SPARK2/metainfo.xml
 bf75f47 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2/services/SPARK/configuration/spark-javaopts-properties.xml
 77a7282 


Diff: https://reviews.apache.org/r/60886/diff/4/

Changes: https://reviews.apache.org/r/60886/diff/3-4/


Testing
---

Verified on local cluster. Unit test for UpgradeCatalog252Test.java passed.


Thanks,

Alejandro Fernandez

Re: Review Request 60886: AMBARI-21481. Upgrading IOP cluster with Spark2 to Ambari 2.5.2 fails on start because config mapping spark2-javaopts-properties is never selected

2017-07-17 Thread Alejandro Fernandez



> On July 17, 2017, 6:26 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog252.java
> > Lines 214-221 (patched)
> > 
> >
> > I don't think this comment is correct - this doesn't deal with a stack 
> > upgrade - it deals with upgrading Ambari, right?
> > 
> > Also - what happens with services which were deleted. Their 
> > configurations remain unselected, no?

This issue is due to Ambari Upgrade, which fails starting the server due to the 
DB consistency checks we have in Ambari 2.5.2


- Alejandro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60886/#review180710
---


On July 17, 2017, 7:12 p.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60886/
> ---
> 
> (Updated July 17, 2017, 7:12 p.m.)
> 
> 
> Review request for Ambari, Andrii Tkach, Dmytro Grinenko, Jonathan Hurley, 
> Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-21481
> https://issues.apache.org/jira/browse/AMBARI-21481
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> STR:
> 
> * Install BigInsights 4.2.5 with Spark2
> * Upgrade to Ambari 2.5.2
> * Start Ambari, which will fail in the DB consistency check
> 
> ```
> 2017-07-07 01:07:01,881 ERROR - You have non selected configs: 
> spark2-javaopts-properties for service SPARK2 from cluster c1!
> ```
> 
> Looks like this config is never selected,
> 
> ```
> ambari=> SELECT sc.service_name, sc.version, sc.user_name, 
> TO_TIMESTAMP(sc.create_timestamp/1000), sc.stack_id, sc.note,
> c.version AS type_version, c.version_tag, c.type_name, c.stack_id, 
> TO_TIMESTAMP(c.create_timestamp/1000) AS clusterconfig_created,
> TO_TIMESTAMP(ccm.create_timestamp/1000) AS clusterconfigmapping_created, 
> ccm.selected, ccm.user_name
> FROM serviceconfig sc JOIN serviceconfigmapping m ON sc.service_config_id = 
> m.service_config_id
> JOIN clusterconfig c ON m.config_id = c.config_id
> JOIN clusterconfigmapping ccm ON c.type_name = ccm.type_name AND 
> c.version_tag = ccm.version_tag
> WHERE sc.service_name = 'SPARK2' and c.type_name = 
> 'spark2-javaopts-properties'
> ORDER BY sc.version, c.type_name;
>  service_name | version |   user_name|  to_timestamp  | stack_id 
> |   note| type_version | version_tag | 
> type_name  | stack_id | clusterconfig_created  | 
> clusterconfigmapping_created | selected | user_name
> --+-+++--+---+--+-++--++--+--+---
>  SPARK2   |   1 | admin  | 2017-07-11 20:17:01+00 |3 
> | Initial configurations for Spark2 |1 | version1| 
> spark2-javaopts-properties |3 | 2017-07-11 20:17:00+00 | 2017-07-11 
> 20:17:01+00   |0 | admin
>  SPARK2   |   2 | ambari-upgrade | 2017-07-13 20:33:50+00 |3 
> |   |1 | version1| 
> spark2-javaopts-properties |3 | 2017-07-11 20:17:00+00 | 2017-07-11 
> 20:17:01+00   |0 | admin
> (2 rows)
> ```
> 
> This is because the config type has a single config with an empty value, so 
> perhaps UI never selects it.
> ```
> 
>   
> content
> Spark2-javaopts-properties
>  
> 
>   
> 
> ```
> 
> Fix will be for Ambari 2.5.2 upgrade to select exactly once config for each 
> config type if none are selected.
> The only config type to hit this issue is spark2-javaopts-properties
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog252.java
>  3c8686c 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/SPARK/configuration/spark-javaopts-properties.xml
>  a197e34 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SPARK2/configuration/spark2-javaopts-properties.xml
>  f8d50fc 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SPARK2/metainfo.xml
>  bf75f47 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2/services/SPARK/configuration/spark-javaopts-properties.xml
>  77a7282 
> 
> 
> Diff: https://reviews.apache.org/r/60886/diff/3/
> 
> 
> Testing
> ---
> 
> Verified on local cluster. Unit test for UpgradeCatalog252Test.java passed.
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Re: Review Request 60886: AMBARI-21481. Upgrading IOP cluster with Spark2 to Ambari 2.5.2 fails on start because config mapping spark2-javaopts-properties is never selected

2017-07-17 Thread Alejandro Fernandez


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60886/
---

(Updated July 17, 2017, 7:12 p.m.)


Review request for Ambari, Andrii Tkach, Dmytro Grinenko, Jonathan Hurley, 
Sumit Mohanty, and Sid Wagle.


Bugs: AMBARI-21481
https://issues.apache.org/jira/browse/AMBARI-21481


Repository: ambari


Description
---

STR:

* Install BigInsights 4.2.5 with Spark2
* Upgrade to Ambari 2.5.2
* Start Ambari, which will fail in the DB consistency check

```
2017-07-07 01:07:01,881 ERROR - You have non selected configs: 
spark2-javaopts-properties for service SPARK2 from cluster c1!
```

Looks like this config is never selected,

```
ambari=> SELECT sc.service_name, sc.version, sc.user_name, 
TO_TIMESTAMP(sc.create_timestamp/1000), sc.stack_id, sc.note,
c.version AS type_version, c.version_tag, c.type_name, c.stack_id, 
TO_TIMESTAMP(c.create_timestamp/1000) AS clusterconfig_created,
TO_TIMESTAMP(ccm.create_timestamp/1000) AS clusterconfigmapping_created, 
ccm.selected, ccm.user_name
FROM serviceconfig sc JOIN serviceconfigmapping m ON sc.service_config_id = 
m.service_config_id
JOIN clusterconfig c ON m.config_id = c.config_id
JOIN clusterconfigmapping ccm ON c.type_name = ccm.type_name AND c.version_tag 
= ccm.version_tag
WHERE sc.service_name = 'SPARK2' and c.type_name = 'spark2-javaopts-properties'
ORDER BY sc.version, c.type_name;
 service_name | version |   user_name|  to_timestamp  | stack_id |  
 note| type_version | version_tag | 
type_name  | stack_id | clusterconfig_created  | 
clusterconfigmapping_created | selected | user_name
--+-+++--+---+--+-++--++--+--+---
 SPARK2   |   1 | admin  | 2017-07-11 20:17:01+00 |3 | 
Initial configurations for Spark2 |1 | version1| 
spark2-javaopts-properties |3 | 2017-07-11 20:17:00+00 | 2017-07-11 
20:17:01+00   |0 | admin
 SPARK2   |   2 | ambari-upgrade | 2017-07-13 20:33:50+00 |3 |  
 |1 | version1| 
spark2-javaopts-properties |3 | 2017-07-11 20:17:00+00 | 2017-07-11 
20:17:01+00   |0 | admin
(2 rows)
```

This is because the config type has a single config with an empty value, so 
perhaps UI never selects it.
```

  
content
Spark2-javaopts-properties
 

  

```

Fix will be for Ambari 2.5.2 upgrade to select exactly once config for each 
config type if none are selected.
The only config type to hit this issue is spark2-javaopts-properties


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog252.java
 3c8686c 
  
ambari-server/src/main/resources/stacks/BigInsights/4.0/services/SPARK/configuration/spark-javaopts-properties.xml
 a197e34 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SPARK2/configuration/spark2-javaopts-properties.xml
 f8d50fc 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SPARK2/metainfo.xml
 bf75f47 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2/services/SPARK/configuration/spark-javaopts-properties.xml
 77a7282 


Diff: https://reviews.apache.org/r/60886/diff/3/

Changes: https://reviews.apache.org/r/60886/diff/2-3/


Testing
---

Verified on local cluster. Unit test for UpgradeCatalog252Test.java passed.


Thanks,

Alejandro Fernandez

Re: Review Request 60886: AMBARI-21481. Upgrading IOP cluster with Spark2 to Ambari 2.5.2 fails on start because config mapping spark2-javaopts-properties is never selected

2017-07-17 Thread Alejandro Fernandez



> On July 15, 2017, 12:40 a.m., Sid Wagle wrote:
> > What happens to configmappings unselected due to service delete? What 
> > wouldbe side effects of selecting those?

Before a service is deleted, it should have exactly one config mapping for a 
certain config type as selected (and in this case that condition is not met). 
If a service is deleted, I need to check if we unselect all of its config 
mappings; but either way it shouldn't matter because if the service is re-added 
we always generate a new config version.


- Alejandro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60886/#review180617
---


On July 15, 2017, 12:20 a.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60886/
> ---
> 
> (Updated July 15, 2017, 12:20 a.m.)
> 
> 
> Review request for Ambari, Andrii Tkach, Dmytro Grinenko, Jonathan Hurley, 
> Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-21481
> https://issues.apache.org/jira/browse/AMBARI-21481
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> STR:
> 
> * Install BigInsights 4.2.5 with Spark2
> * Upgrade to Ambari 2.5.2
> * Start Ambari, which will fail in the DB consistency check
> 
> ```
> 2017-07-07 01:07:01,881 ERROR - You have non selected configs: 
> spark2-javaopts-properties for service SPARK2 from cluster c1!
> ```
> 
> Looks like this config is never selected,
> 
> ```
> ambari=> SELECT sc.service_name, sc.version, sc.user_name, 
> TO_TIMESTAMP(sc.create_timestamp/1000), sc.stack_id, sc.note,
> c.version AS type_version, c.version_tag, c.type_name, c.stack_id, 
> TO_TIMESTAMP(c.create_timestamp/1000) AS clusterconfig_created,
> TO_TIMESTAMP(ccm.create_timestamp/1000) AS clusterconfigmapping_created, 
> ccm.selected, ccm.user_name
> FROM serviceconfig sc JOIN serviceconfigmapping m ON sc.service_config_id = 
> m.service_config_id
> JOIN clusterconfig c ON m.config_id = c.config_id
> JOIN clusterconfigmapping ccm ON c.type_name = ccm.type_name AND 
> c.version_tag = ccm.version_tag
> WHERE sc.service_name = 'SPARK2' and c.type_name = 
> 'spark2-javaopts-properties'
> ORDER BY sc.version, c.type_name;
>  service_name | version |   user_name|  to_timestamp  | stack_id 
> |   note| type_version | version_tag | 
> type_name  | stack_id | clusterconfig_created  | 
> clusterconfigmapping_created | selected | user_name
> --+-+++--+---+--+-++--++--+--+---
>  SPARK2   |   1 | admin  | 2017-07-11 20:17:01+00 |3 
> | Initial configurations for Spark2 |1 | version1| 
> spark2-javaopts-properties |3 | 2017-07-11 20:17:00+00 | 2017-07-11 
> 20:17:01+00   |0 | admin
>  SPARK2   |   2 | ambari-upgrade | 2017-07-13 20:33:50+00 |3 
> |   |1 | version1| 
> spark2-javaopts-properties |3 | 2017-07-11 20:17:00+00 | 2017-07-11 
> 20:17:01+00   |0 | admin
> (2 rows)
> ```
> 
> This is because the config type has a single config with an empty value, so 
> perhaps UI never selects it.
> ```
> 
>   
> content
> Spark2-javaopts-properties
>  
> 
>   
> 
> ```
> 
> Fix will be for Ambari 2.5.2 upgrade to select exactly once config for each 
> config type if none are selected.
> The only config type to hit this issue is spark2-javaopts-properties
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog252.java
>  6d2ab84 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/SPARK/configuration/spark-javaopts-properties.xml
>  a197e34 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SPARK2/configuration/spark2-javaopts-properties.xml
>  f8d50fc 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SPARK2/metainfo.xml
>  bf75f47 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2/services/SPARK/configuration/spark-javaopts-properties.xml
>  77a7282 
> 
> 
> Diff: https://reviews.apache.org/r/60886/diff/2/
> 
> 
> Testing
> ---
> 
> Verified on local cluster. Unit test for UpgradeCatalog252Test.java passed.
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Re: Review Request 60886: AMBARI-21481. Upgrading IOP cluster with Spark2 to Ambari 2.5.2 fails on start because config mapping spark2-javaopts-properties is never selected

2017-07-17 Thread Alejandro Fernandez



> On July 17, 2017, 3:29 p.m., Jonathan Hurley wrote:
> > So, this is an env file with no content? Doesn't that mean that it's 
> > optional and doesn't need to be created? Can we simply use the  
> > tag (see AMBARI-21364) and have the DB checker skip it?

The config is already saved to the DB consistency checker applies its business 
logic that at least one config mapping must be selected. If this were HDP, I 
could also mark it as  but since it's for the BigInsights stack it 
won't affect any new clusters.


- Alejandro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60886/#review180681
---


On July 15, 2017, 12:20 a.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60886/
> ---
> 
> (Updated July 15, 2017, 12:20 a.m.)
> 
> 
> Review request for Ambari, Andrii Tkach, Dmytro Grinenko, Jonathan Hurley, 
> Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-21481
> https://issues.apache.org/jira/browse/AMBARI-21481
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> STR:
> 
> * Install BigInsights 4.2.5 with Spark2
> * Upgrade to Ambari 2.5.2
> * Start Ambari, which will fail in the DB consistency check
> 
> ```
> 2017-07-07 01:07:01,881 ERROR - You have non selected configs: 
> spark2-javaopts-properties for service SPARK2 from cluster c1!
> ```
> 
> Looks like this config is never selected,
> 
> ```
> ambari=> SELECT sc.service_name, sc.version, sc.user_name, 
> TO_TIMESTAMP(sc.create_timestamp/1000), sc.stack_id, sc.note,
> c.version AS type_version, c.version_tag, c.type_name, c.stack_id, 
> TO_TIMESTAMP(c.create_timestamp/1000) AS clusterconfig_created,
> TO_TIMESTAMP(ccm.create_timestamp/1000) AS clusterconfigmapping_created, 
> ccm.selected, ccm.user_name
> FROM serviceconfig sc JOIN serviceconfigmapping m ON sc.service_config_id = 
> m.service_config_id
> JOIN clusterconfig c ON m.config_id = c.config_id
> JOIN clusterconfigmapping ccm ON c.type_name = ccm.type_name AND 
> c.version_tag = ccm.version_tag
> WHERE sc.service_name = 'SPARK2' and c.type_name = 
> 'spark2-javaopts-properties'
> ORDER BY sc.version, c.type_name;
>  service_name | version |   user_name|  to_timestamp  | stack_id 
> |   note| type_version | version_tag | 
> type_name  | stack_id | clusterconfig_created  | 
> clusterconfigmapping_created | selected | user_name
> --+-+++--+---+--+-++--++--+--+---
>  SPARK2   |   1 | admin  | 2017-07-11 20:17:01+00 |3 
> | Initial configurations for Spark2 |1 | version1| 
> spark2-javaopts-properties |3 | 2017-07-11 20:17:00+00 | 2017-07-11 
> 20:17:01+00   |0 | admin
>  SPARK2   |   2 | ambari-upgrade | 2017-07-13 20:33:50+00 |3 
> |   |1 | version1| 
> spark2-javaopts-properties |3 | 2017-07-11 20:17:00+00 | 2017-07-11 
> 20:17:01+00   |0 | admin
> (2 rows)
> ```
> 
> This is because the config type has a single config with an empty value, so 
> perhaps UI never selects it.
> ```
> 
>   
> content
> Spark2-javaopts-properties
>  
> 
>   
> 
> ```
> 
> Fix will be for Ambari 2.5.2 upgrade to select exactly once config for each 
> config type if none are selected.
> The only config type to hit this issue is spark2-javaopts-properties
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog252.java
>  6d2ab84 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/SPARK/configuration/spark-javaopts-properties.xml
>  a197e34 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SPARK2/configuration/spark2-javaopts-properties.xml
>  f8d50fc 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SPARK2/metainfo.xml
>  bf75f47 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2/services/SPARK/configuration/spark-javaopts-properties.xml
>  77a7282 
> 
> 
> Diff: https://reviews.apache.org/r/60886/diff/2/
> 
> 
> Testing
> ---
> 
> Verified on local cluster. Unit test for UpgradeCatalog252Test.java passed.
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Re: Review Request 60886: AMBARI-21481. Upgrading IOP cluster with Spark2 to Ambari 2.5.2 fails on start because config mapping spark2-javaopts-properties is never selected

2017-07-17 Thread Jonathan Hurley


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60886/#review180681
---



So, this is an env file with no content? Doesn't that mean that it's optional 
and doesn't need to be created? Can we simply use the  tag (see 
AMBARI-21364) and have the DB checker skip it?

- Jonathan Hurley


On July 14, 2017, 8:20 p.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60886/
> ---
> 
> (Updated July 14, 2017, 8:20 p.m.)
> 
> 
> Review request for Ambari, Andrii Tkach, Dmytro Grinenko, Jonathan Hurley, 
> Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-21481
> https://issues.apache.org/jira/browse/AMBARI-21481
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> STR:
> 
> * Install BigInsights 4.2.5 with Spark2
> * Upgrade to Ambari 2.5.2
> * Start Ambari, which will fail in the DB consistency check
> 
> ```
> 2017-07-07 01:07:01,881 ERROR - You have non selected configs: 
> spark2-javaopts-properties for service SPARK2 from cluster c1!
> ```
> 
> Looks like this config is never selected,
> 
> ```
> ambari=> SELECT sc.service_name, sc.version, sc.user_name, 
> TO_TIMESTAMP(sc.create_timestamp/1000), sc.stack_id, sc.note,
> c.version AS type_version, c.version_tag, c.type_name, c.stack_id, 
> TO_TIMESTAMP(c.create_timestamp/1000) AS clusterconfig_created,
> TO_TIMESTAMP(ccm.create_timestamp/1000) AS clusterconfigmapping_created, 
> ccm.selected, ccm.user_name
> FROM serviceconfig sc JOIN serviceconfigmapping m ON sc.service_config_id = 
> m.service_config_id
> JOIN clusterconfig c ON m.config_id = c.config_id
> JOIN clusterconfigmapping ccm ON c.type_name = ccm.type_name AND 
> c.version_tag = ccm.version_tag
> WHERE sc.service_name = 'SPARK2' and c.type_name = 
> 'spark2-javaopts-properties'
> ORDER BY sc.version, c.type_name;
>  service_name | version |   user_name|  to_timestamp  | stack_id 
> |   note| type_version | version_tag | 
> type_name  | stack_id | clusterconfig_created  | 
> clusterconfigmapping_created | selected | user_name
> --+-+++--+---+--+-++--++--+--+---
>  SPARK2   |   1 | admin  | 2017-07-11 20:17:01+00 |3 
> | Initial configurations for Spark2 |1 | version1| 
> spark2-javaopts-properties |3 | 2017-07-11 20:17:00+00 | 2017-07-11 
> 20:17:01+00   |0 | admin
>  SPARK2   |   2 | ambari-upgrade | 2017-07-13 20:33:50+00 |3 
> |   |1 | version1| 
> spark2-javaopts-properties |3 | 2017-07-11 20:17:00+00 | 2017-07-11 
> 20:17:01+00   |0 | admin
> (2 rows)
> ```
> 
> This is because the config type has a single config with an empty value, so 
> perhaps UI never selects it.
> ```
> 
>   
> content
> Spark2-javaopts-properties
>  
> 
>   
> 
> ```
> 
> Fix will be for Ambari 2.5.2 upgrade to select exactly once config for each 
> config type if none are selected.
> The only config type to hit this issue is spark2-javaopts-properties
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog252.java
>  6d2ab84 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/SPARK/configuration/spark-javaopts-properties.xml
>  a197e34 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SPARK2/configuration/spark2-javaopts-properties.xml
>  f8d50fc 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SPARK2/metainfo.xml
>  bf75f47 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2/services/SPARK/configuration/spark-javaopts-properties.xml
>  77a7282 
> 
> 
> Diff: https://reviews.apache.org/r/60886/diff/2/
> 
> 
> Testing
> ---
> 
> Verified on local cluster. Unit test for UpgradeCatalog252Test.java passed.
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Re: Review Request 60886: AMBARI-21481. Upgrading IOP cluster with Spark2 to Ambari 2.5.2 fails on start because config mapping spark2-javaopts-properties is never selected

2017-07-14 Thread Sid Wagle


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60886/#review180617
---



What happens to configmappings unselected due to service delete? What wouldbe 
side effects of selecting those?

- Sid Wagle


On July 15, 2017, 12:20 a.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60886/
> ---
> 
> (Updated July 15, 2017, 12:20 a.m.)
> 
> 
> Review request for Ambari, Andrii Tkach, Dmytro Grinenko, Jonathan Hurley, 
> Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-21481
> https://issues.apache.org/jira/browse/AMBARI-21481
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> STR:
> 
> * Install BigInsights 4.2.5 with Spark2
> * Upgrade to Ambari 2.5.2
> * Start Ambari, which will fail in the DB consistency check
> 
> ```
> 2017-07-07 01:07:01,881 ERROR - You have non selected configs: 
> spark2-javaopts-properties for service SPARK2 from cluster c1!
> ```
> 
> Looks like this config is never selected,
> 
> ```
> ambari=> SELECT sc.service_name, sc.version, sc.user_name, 
> TO_TIMESTAMP(sc.create_timestamp/1000), sc.stack_id, sc.note,
> c.version AS type_version, c.version_tag, c.type_name, c.stack_id, 
> TO_TIMESTAMP(c.create_timestamp/1000) AS clusterconfig_created,
> TO_TIMESTAMP(ccm.create_timestamp/1000) AS clusterconfigmapping_created, 
> ccm.selected, ccm.user_name
> FROM serviceconfig sc JOIN serviceconfigmapping m ON sc.service_config_id = 
> m.service_config_id
> JOIN clusterconfig c ON m.config_id = c.config_id
> JOIN clusterconfigmapping ccm ON c.type_name = ccm.type_name AND 
> c.version_tag = ccm.version_tag
> WHERE sc.service_name = 'SPARK2' and c.type_name = 
> 'spark2-javaopts-properties'
> ORDER BY sc.version, c.type_name;
>  service_name | version |   user_name|  to_timestamp  | stack_id 
> |   note| type_version | version_tag | 
> type_name  | stack_id | clusterconfig_created  | 
> clusterconfigmapping_created | selected | user_name
> --+-+++--+---+--+-++--++--+--+---
>  SPARK2   |   1 | admin  | 2017-07-11 20:17:01+00 |3 
> | Initial configurations for Spark2 |1 | version1| 
> spark2-javaopts-properties |3 | 2017-07-11 20:17:00+00 | 2017-07-11 
> 20:17:01+00   |0 | admin
>  SPARK2   |   2 | ambari-upgrade | 2017-07-13 20:33:50+00 |3 
> |   |1 | version1| 
> spark2-javaopts-properties |3 | 2017-07-11 20:17:00+00 | 2017-07-11 
> 20:17:01+00   |0 | admin
> (2 rows)
> ```
> 
> This is because the config type has a single config with an empty value, so 
> perhaps UI never selects it.
> ```
> 
>   
> content
> Spark2-javaopts-properties
>  
> 
>   
> 
> ```
> 
> Fix will be for Ambari 2.5.2 upgrade to select exactly once config for each 
> config type if none are selected.
> The only config type to hit this issue is spark2-javaopts-properties
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog252.java
>  6d2ab84 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/SPARK/configuration/spark-javaopts-properties.xml
>  a197e34 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SPARK2/configuration/spark2-javaopts-properties.xml
>  f8d50fc 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SPARK2/metainfo.xml
>  bf75f47 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2/services/SPARK/configuration/spark-javaopts-properties.xml
>  77a7282 
> 
> 
> Diff: https://reviews.apache.org/r/60886/diff/2/
> 
> 
> Testing
> ---
> 
> Verified on local cluster. Unit test for UpgradeCatalog252Test.java passed.
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Re: Review Request 60886: AMBARI-21481. Upgrading IOP cluster with Spark2 to Ambari 2.5.2 fails on start because config mapping spark2-javaopts-properties is never selected

2017-07-14 Thread Alejandro Fernandez


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60886/
---

(Updated July 15, 2017, 12:20 a.m.)


Review request for Ambari, Andrii Tkach, Dmytro Grinenko, Jonathan Hurley, 
Sumit Mohanty, and Sid Wagle.


Changes
---

Added java doc


Bugs: AMBARI-21481
https://issues.apache.org/jira/browse/AMBARI-21481


Repository: ambari


Description
---

STR:

* Install BigInsights 4.2.5 with Spark2
* Upgrade to Ambari 2.5.2
* Start Ambari, which will fail in the DB consistency check

```
2017-07-07 01:07:01,881 ERROR - You have non selected configs: 
spark2-javaopts-properties for service SPARK2 from cluster c1!
```

Looks like this config is never selected,

```
ambari=> SELECT sc.service_name, sc.version, sc.user_name, 
TO_TIMESTAMP(sc.create_timestamp/1000), sc.stack_id, sc.note,
c.version AS type_version, c.version_tag, c.type_name, c.stack_id, 
TO_TIMESTAMP(c.create_timestamp/1000) AS clusterconfig_created,
TO_TIMESTAMP(ccm.create_timestamp/1000) AS clusterconfigmapping_created, 
ccm.selected, ccm.user_name
FROM serviceconfig sc JOIN serviceconfigmapping m ON sc.service_config_id = 
m.service_config_id
JOIN clusterconfig c ON m.config_id = c.config_id
JOIN clusterconfigmapping ccm ON c.type_name = ccm.type_name AND c.version_tag 
= ccm.version_tag
WHERE sc.service_name = 'SPARK2' and c.type_name = 'spark2-javaopts-properties'
ORDER BY sc.version, c.type_name;
 service_name | version |   user_name|  to_timestamp  | stack_id |  
 note| type_version | version_tag | 
type_name  | stack_id | clusterconfig_created  | 
clusterconfigmapping_created | selected | user_name
--+-+++--+---+--+-++--++--+--+---
 SPARK2   |   1 | admin  | 2017-07-11 20:17:01+00 |3 | 
Initial configurations for Spark2 |1 | version1| 
spark2-javaopts-properties |3 | 2017-07-11 20:17:00+00 | 2017-07-11 
20:17:01+00   |0 | admin
 SPARK2   |   2 | ambari-upgrade | 2017-07-13 20:33:50+00 |3 |  
 |1 | version1| 
spark2-javaopts-properties |3 | 2017-07-11 20:17:00+00 | 2017-07-11 
20:17:01+00   |0 | admin
(2 rows)
```

This is because the config type has a single config with an empty value, so 
perhaps UI never selects it.
```

  
content
Spark2-javaopts-properties
 

  

```

Fix will be for Ambari 2.5.2 upgrade to select exactly once config for each 
config type if none are selected.
The only config type to hit this issue is spark2-javaopts-properties


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog252.java
 6d2ab84 
  
ambari-server/src/main/resources/stacks/BigInsights/4.0/services/SPARK/configuration/spark-javaopts-properties.xml
 a197e34 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SPARK2/configuration/spark2-javaopts-properties.xml
 f8d50fc 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SPARK2/metainfo.xml
 bf75f47 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2/services/SPARK/configuration/spark-javaopts-properties.xml
 77a7282 


Diff: https://reviews.apache.org/r/60886/diff/2/

Changes: https://reviews.apache.org/r/60886/diff/1-2/


Testing
---

Verified on local cluster. Unit test for UpgradeCatalog252Test.java passed.


Thanks,

Alejandro Fernandez

Re: Review Request 60886: AMBARI-21481. Upgrading IOP cluster with Spark2 to Ambari 2.5.2 fails on start because config mapping spark2-javaopts-properties is never selected

2017-07-14 Thread Alejandro Fernandez


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60886/#review180614
---




ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog252.java
Lines 214 (patched)


Will add javadoc.



ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog252.java
Lines 261 (patched)


The only config type that had this issue was spark2-javaopts-properties so 
I feel confident keeping it generic.



ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SPARK2/configuration/spark2-javaopts-properties.xml
Lines 27 (patched)


Will raise a Jira for Ambari 2.5.3 for the UI to select config types that 
only contain configs with empty values.


- Alejandro Fernandez


On July 15, 2017, 12:14 a.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60886/
> ---
> 
> (Updated July 15, 2017, 12:14 a.m.)
> 
> 
> Review request for Ambari, Andrii Tkach, Dmytro Grinenko, Jonathan Hurley, 
> Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-21481
> https://issues.apache.org/jira/browse/AMBARI-21481
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> STR:
> 
> * Install BigInsights 4.2.5 with Spark2
> * Upgrade to Ambari 2.5.2
> * Start Ambari, which will fail in the DB consistency check
> 
> ```
> 2017-07-07 01:07:01,881 ERROR - You have non selected configs: 
> spark2-javaopts-properties for service SPARK2 from cluster c1!
> ```
> 
> Looks like this config is never selected,
> 
> ```
> ambari=> SELECT sc.service_name, sc.version, sc.user_name, 
> TO_TIMESTAMP(sc.create_timestamp/1000), sc.stack_id, sc.note,
> c.version AS type_version, c.version_tag, c.type_name, c.stack_id, 
> TO_TIMESTAMP(c.create_timestamp/1000) AS clusterconfig_created,
> TO_TIMESTAMP(ccm.create_timestamp/1000) AS clusterconfigmapping_created, 
> ccm.selected, ccm.user_name
> FROM serviceconfig sc JOIN serviceconfigmapping m ON sc.service_config_id = 
> m.service_config_id
> JOIN clusterconfig c ON m.config_id = c.config_id
> JOIN clusterconfigmapping ccm ON c.type_name = ccm.type_name AND 
> c.version_tag = ccm.version_tag
> WHERE sc.service_name = 'SPARK2' and c.type_name = 
> 'spark2-javaopts-properties'
> ORDER BY sc.version, c.type_name;
>  service_name | version |   user_name|  to_timestamp  | stack_id 
> |   note| type_version | version_tag | 
> type_name  | stack_id | clusterconfig_created  | 
> clusterconfigmapping_created | selected | user_name
> --+-+++--+---+--+-++--++--+--+---
>  SPARK2   |   1 | admin  | 2017-07-11 20:17:01+00 |3 
> | Initial configurations for Spark2 |1 | version1| 
> spark2-javaopts-properties |3 | 2017-07-11 20:17:00+00 | 2017-07-11 
> 20:17:01+00   |0 | admin
>  SPARK2   |   2 | ambari-upgrade | 2017-07-13 20:33:50+00 |3 
> |   |1 | version1| 
> spark2-javaopts-properties |3 | 2017-07-11 20:17:00+00 | 2017-07-11 
> 20:17:01+00   |0 | admin
> (2 rows)
> ```
> 
> This is because the config type has a single config with an empty value, so 
> perhaps UI never selects it.
> ```
> 
>   
> content
> Spark2-javaopts-properties
>  
> 
>   
> 
> ```
> 
> Fix will be for Ambari 2.5.2 upgrade to select exactly once config for each 
> config type if none are selected.
> The only config type to hit this issue is spark2-javaopts-properties
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog252.java
>  6d2ab84 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.0/services/SPARK/configuration/spark-javaopts-properties.xml
>  a197e34 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SPARK2/configuration/spark2-javaopts-properties.xml
>  f8d50fc 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SPARK2/metainfo.xml
>  bf75f47 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2/services/SPARK/configuration/spark-javaopts-properties.xml
>  77a7282 
> 
> 
> Diff: https://reviews.apache.org/r/60886/diff/1/
> 
> 
> Testing
> ---
> 
> Verified on local cluster. Unit test for UpgradeCatalog252Test.java passed.
> 
> 
>

Re: Review Request 60886: AMBARI-21481. Upgrading IOP cluster with Spark2 to Ambari 2.5.2 fails on start because config mapping spark2-javaopts-properties is never selected

Re: Review Request 60886: AMBARI-21481. Upgrading IOP cluster with Spark2 to Ambari 2.5.2 fails on start because config mapping spark2-javaopts-properties is never selected

Re: Review Request 60886: AMBARI-21481. Upgrading IOP cluster with Spark2 to Ambari 2.5.2 fails on start because config mapping spark2-javaopts-properties is never selected

Re: Review Request 60886: AMBARI-21481. Upgrading IOP cluster with Spark2 to Ambari 2.5.2 fails on start because config mapping spark2-javaopts-properties is never selected

Re: Review Request 60886: AMBARI-21481. Upgrading IOP cluster with Spark2 to Ambari 2.5.2 fails on start because config mapping spark2-javaopts-properties is never selected

Re: Review Request 60886: AMBARI-21481. Upgrading IOP cluster with Spark2 to Ambari 2.5.2 fails on start because config mapping spark2-javaopts-properties is never selected

Re: Review Request 60886: AMBARI-21481. Upgrading IOP cluster with Spark2 to Ambari 2.5.2 fails on start because config mapping spark2-javaopts-properties is never selected

Re: Review Request 60886: AMBARI-21481. Upgrading IOP cluster with Spark2 to Ambari 2.5.2 fails on start because config mapping spark2-javaopts-properties is never selected

Re: Review Request 60886: AMBARI-21481. Upgrading IOP cluster with Spark2 to Ambari 2.5.2 fails on start because config mapping spark2-javaopts-properties is never selected

Re: Review Request 60886: AMBARI-21481. Upgrading IOP cluster with Spark2 to Ambari 2.5.2 fails on start because config mapping spark2-javaopts-properties is never selected

Re: Review Request 60886: AMBARI-21481. Upgrading IOP cluster with Spark2 to Ambari 2.5.2 fails on start because config mapping spark2-javaopts-properties is never selected

Re: Review Request 60886: AMBARI-21481. Upgrading IOP cluster with Spark2 to Ambari 2.5.2 fails on start because config mapping spark2-javaopts-properties is never selected

12 matches

Site Navigation

Mail list logo

Footer information