date:20170705

Re: Review Request 60655: AMBARI-21405. Create custom action to force-remove packages

2017-07-05 Thread Attila Doroszlai



> On July 5, 2017, 7:42 p.m., Jonathan Hurley wrote:
> > ambari-common/src/main/python/resource_management/core/providers/package/apt.py
> > Line 106 (original), 107 (patched)
> > 
> >
> > For clarity, can we change this naming convention from "with" to 
> > "ignore". Using "with" makes it seem like you're also removing 
> > dependencies. A simple ignore_dependencies=False would be good.

In case of `with_deps = True`, the package manager in fact was removing 
dependencies, so that was correct.  Anyway I changed it to 
`ignore_dependencies` for clarity.


- Attila


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60655/#review179663
---


On July 6, 2017, 8:32 a.m., Attila Doroszlai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60655/
> ---
> 
> (Updated July 6, 2017, 8:32 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Sumit 
> Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-21405
> https://issues.apache.org/jira/browse/AMBARI-21405
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> * Add option in `Package` resource to remove package while leaving its 
> dependencies installed
> * Add custom action `force_remove_packages`
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/test/python/resource_management/TestPackageResource.py 
> 66227c66a7768aa044f30a1c76d782c5101be678 
>   
> ambari-common/src/main/python/resource_management/core/providers/package/__init__.py
>  21de1839c853794f252a4cb2265b89549a802d88 
>   
> ambari-common/src/main/python/resource_management/core/providers/package/apt.py
>  d095173f07c46842b6e9e38dc933e06a3e43a749 
>   
> ambari-common/src/main/python/resource_management/core/providers/package/choco.py
>  db552964279f509de90d2fde884bfc0fea12f7b0 
>   
> ambari-common/src/main/python/resource_management/core/providers/package/yumrpm.py
>  ea10a8601430b62bbe216848e1d26a5e8bc16927 
>   
> ambari-common/src/main/python/resource_management/core/providers/package/zypper.py
>  265c1626873bfb98196f102e73bb2dbedee94c80 
>   
> ambari-common/src/main/python/resource_management/core/resources/packaging.py 
> e3adc30f46abd3f33670f817421d88270b3e7ade 
>   
> ambari-server/src/main/resources/custom_action_definitions/system_action_definitions.xml
>  0f502568ddfb5da7d5574b2e33c6a354c37d9697 
>   
> ambari-server/src/main/resources/custom_actions/scripts/force_remove_packages.py
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/60655/diff/2/
> 
> 
> Testing
> ---
> 
> Tested custom action as part of cross-stack upgrade on CentOS7.
> 
> Manually tested custom action on Debian 7.6:
> 
> ```
> $ curl -X POST -d '{ "RequestInfo": { "action": "force_remove_packages", 
> "parameters": { "package_list": "vim-common" } } }' 
> http://localhost:8080/api/v1/clusters/TEST/requests
> 
> output.txt:
> 2017-07-05 12:29:59,400 - Package['vim-common'] {'action': ['remove'], 
> 'remove_with_dependencies': False}
> 2017-07-05 12:29:59,411 - Removing package vim-common ('/usr/bin/dpkg 
> --remove --ignore-depends vim-common vim-common')
> 2017-07-05 12:30:00,702 - Removed vim-common
> 
> $ apt-get install vim
> ...
> vim is already the newest version.
> You might want to run 'apt-get -f install' to correct these:
> The following packages have unmet dependencies:
>  vim : Depends: vim-common (= 2:7.3.547-7+deb7u3) but it is not going to be 
> installed
> ```
> 
> New unit tests for `Package` in `ambari-agent`:
> 
> ```
> Ran 456 tests in 38.256s
> 
> OK
> ```
> 
> 
> Thanks,
> 
> Attila Doroszlai
> 
>

Re: Review Request 60655: AMBARI-21405. Create custom action to force-remove packages

2017-07-05 Thread Attila Doroszlai


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60655/
---

(Updated July 6, 2017, 8:32 a.m.)


Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Sumit Mohanty, 
and Sid Wagle.


Changes
---

* changed option to `ignore_dependencies`
* added doc
* added package list in failure message


Bugs: AMBARI-21405
https://issues.apache.org/jira/browse/AMBARI-21405


Repository: ambari


Description
---

* Add option in `Package` resource to remove package while leaving its 
dependencies installed
* Add custom action `force_remove_packages`


Diffs (updated)
-

  ambari-agent/src/test/python/resource_management/TestPackageResource.py 
66227c66a7768aa044f30a1c76d782c5101be678 
  
ambari-common/src/main/python/resource_management/core/providers/package/__init__.py
 21de1839c853794f252a4cb2265b89549a802d88 
  
ambari-common/src/main/python/resource_management/core/providers/package/apt.py 
d095173f07c46842b6e9e38dc933e06a3e43a749 
  
ambari-common/src/main/python/resource_management/core/providers/package/choco.py
 db552964279f509de90d2fde884bfc0fea12f7b0 
  
ambari-common/src/main/python/resource_management/core/providers/package/yumrpm.py
 ea10a8601430b62bbe216848e1d26a5e8bc16927 
  
ambari-common/src/main/python/resource_management/core/providers/package/zypper.py
 265c1626873bfb98196f102e73bb2dbedee94c80 
  ambari-common/src/main/python/resource_management/core/resources/packaging.py 
e3adc30f46abd3f33670f817421d88270b3e7ade 
  
ambari-server/src/main/resources/custom_action_definitions/system_action_definitions.xml
 0f502568ddfb5da7d5574b2e33c6a354c37d9697 
  
ambari-server/src/main/resources/custom_actions/scripts/force_remove_packages.py
 PRE-CREATION 


Diff: https://reviews.apache.org/r/60655/diff/2/

Changes: https://reviews.apache.org/r/60655/diff/1-2/


Testing
---

Tested custom action as part of cross-stack upgrade on CentOS7.

Manually tested custom action on Debian 7.6:

```
$ curl -X POST -d '{ "RequestInfo": { "action": "force_remove_packages", 
"parameters": { "package_list": "vim-common" } } }' 
http://localhost:8080/api/v1/clusters/TEST/requests

output.txt:
2017-07-05 12:29:59,400 - Package['vim-common'] {'action': ['remove'], 
'remove_with_dependencies': False}
2017-07-05 12:29:59,411 - Removing package vim-common ('/usr/bin/dpkg --remove 
--ignore-depends vim-common vim-common')
2017-07-05 12:30:00,702 - Removed vim-common

$ apt-get install vim
...
vim is already the newest version.
You might want to run 'apt-get -f install' to correct these:
The following packages have unmet dependencies:
 vim : Depends: vim-common (= 2:7.3.547-7+deb7u3) but it is not going to be 
installed
```

New unit tests for `Package` in `ambari-agent`:

```
Ran 456 tests in 38.256s

OK
```


Thanks,

Attila Doroszlai

Re: Review Request 60574: AMBARI-21383 Update upgrade pack to set mariadb support property for hive

2017-07-05 Thread Jonathan Hurley


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60574/#review179752
---


Ship it!




Ship It!

- Jonathan Hurley


On June 30, 2017, 3:15 p.m., Di Li wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60574/
> ---
> 
> (Updated June 30, 2017, 3:15 p.m.)
> 
> 
> Review request for Ambari, Dmytro Grinenko, Di Li, Dmitro Lisnichenko, 
> Jonathan Hurley, Sumit Mohanty, Sid Wagle, and Tim Thorpe.
> 
> 
> Bugs: AMBARI-21383
> https://issues.apache.org/jira/browse/AMBARI-21383
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Update upgrade pack to set mariadb support property for hive
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/config-upgrade.xml
>  540c017 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
>  e8a1e33 
> 
> 
> Diff: https://reviews.apache.org/r/60574/diff/1/
> 
> 
> Testing
> ---
> 
> integration test later on.
> 
> 
> Thanks,
> 
> Di Li
> 
>

Re: Review Request 60671: AMBARI-21411. Backend - Run EU/RU PreChecks during a cross-stack upgrade

2017-07-05 Thread Jonathan Hurley


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60671/#review179751
---


Ship it!




Ship It!

- Jonathan Hurley


On July 5, 2017, 8:21 p.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60671/
> ---
> 
> (Updated July 5, 2017, 8:21 p.m.)
> 
> 
> Review request for Ambari, Attila Doroszlai, Dmytro Grinenko, Jonathan 
> Hurley, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-21411
> https://issues.apache.org/jira/browse/AMBARI-21411
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> During a cross-stack upgrade, the PreChecks need to be able to run.
> 
> STR:
> 
> * Install BigInsights 4.2.5
> * Upgrade Ambari to 2.5
> * Register bits for HDP 2.6
> * Install bits for HDP 2.6
> * Attempt to run PreChecks
> 
> ```
> curl -u admin:admin -H "X-Requested-By: ambari" -i -X GET 
> http://localhost:8080/api/v1/clusters/c1/rolling_upgrades_check?fields=*&UpgradeChecks/repository_version=2.6.1.0-129&UpgradeChecks/upgrade_type=NON_ROLLING&UpgradeChecks/upgrade_pack=nonrolling-upgrade-to-hdp-2.6
> ```
> 
> This returns,
> ```
> {
>   "status" : 500,
>   "message" : "org.apache.ambari.server.controller.spi.SystemException: 
> Repository version null was not found"
> }
> ```
> 
> because in UpgradeHelper.java it cannot find the upgrade pack.
> 
> The new API will requiring passing in a new argument, e.g., 
> UpgradeChecks/target_stack=HDP-2.6
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/PreUpgradeCheckResourceProvider.java
>  7ccafb7 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java
>  795e8d1 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java 
> 69663fb 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/FLUME/metainfo.xml
>  99f1f79 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/JNBG/metainfo.xml
>  3520a32 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/metainfo.xml
>  4a7dc31 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/R4ML/metainfo.xml
>  d1c708d 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/metainfo.xml
>  be20810 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SYSTEMML/metainfo.xml
>  7a0e125 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/TITAN/metainfo.xml
>  d00e707 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/PreUpgradeCheckResourceProviderTest.java
>  47bba34 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/UpgradeHelperTest.java
>  6e27cba 
> 
> 
> Diff: https://reviews.apache.org/r/60671/diff/1/
> 
> 
> Testing
> ---
> 
> Verified on local cluster
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Review Request 60676: Cluster Effective Version Can Be Cached Incorrectly During an Upgrade

2017-07-05 Thread Jonathan Hurley


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60676/
---

Review request for Ambari, Alejandro Fernandez and Sid Wagle.


Bugs: AMBARI-21412
https://issues.apache.org/jira/browse/AMBARI-21412


Repository: ambari


Description
---

When performing an upgrade, the cluster effective version can be cached 
incorrectly with the old version after the {{UpdateDesiredStackAction}} 
invalidates the cache. The cache should check to see if the 
{{UpdateDesiredStackAction}} is in progress and choose not to cache the value 
at this time.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/UpdateDesiredStackAction.java
 b071914a49 
  
ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
 db67807610 


Diff: https://reviews.apache.org/r/60676/diff/1/


Testing
---

PENDING


Thanks,

Jonathan Hurley

Re: Review Request 60671: AMBARI-21411. Backend - Run EU/RU PreChecks during a cross-stack upgrade

2017-07-05 Thread Sid Wagle


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60671/#review179742
---


Ship it!




Ship It!

- Sid Wagle


On July 6, 2017, 12:21 a.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60671/
> ---
> 
> (Updated July 6, 2017, 12:21 a.m.)
> 
> 
> Review request for Ambari, Attila Doroszlai, Dmytro Grinenko, Jonathan 
> Hurley, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-21411
> https://issues.apache.org/jira/browse/AMBARI-21411
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> During a cross-stack upgrade, the PreChecks need to be able to run.
> 
> STR:
> 
> * Install BigInsights 4.2.5
> * Upgrade Ambari to 2.5
> * Register bits for HDP 2.6
> * Install bits for HDP 2.6
> * Attempt to run PreChecks
> 
> ```
> curl -u admin:admin -H "X-Requested-By: ambari" -i -X GET 
> http://localhost:8080/api/v1/clusters/c1/rolling_upgrades_check?fields=*&UpgradeChecks/repository_version=2.6.1.0-129&UpgradeChecks/upgrade_type=NON_ROLLING&UpgradeChecks/upgrade_pack=nonrolling-upgrade-to-hdp-2.6
> ```
> 
> This returns,
> ```
> {
>   "status" : 500,
>   "message" : "org.apache.ambari.server.controller.spi.SystemException: 
> Repository version null was not found"
> }
> ```
> 
> because in UpgradeHelper.java it cannot find the upgrade pack.
> 
> The new API will requiring passing in a new argument, e.g., 
> UpgradeChecks/target_stack=HDP-2.6
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/PreUpgradeCheckResourceProvider.java
>  7ccafb7 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java
>  795e8d1 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java 
> 69663fb 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/FLUME/metainfo.xml
>  99f1f79 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/JNBG/metainfo.xml
>  3520a32 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/metainfo.xml
>  4a7dc31 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/R4ML/metainfo.xml
>  d1c708d 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/metainfo.xml
>  be20810 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SYSTEMML/metainfo.xml
>  7a0e125 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/TITAN/metainfo.xml
>  d00e707 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/PreUpgradeCheckResourceProviderTest.java
>  47bba34 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/UpgradeHelperTest.java
>  6e27cba 
> 
> 
> Diff: https://reviews.apache.org/r/60671/diff/1/
> 
> 
> Testing
> ---
> 
> Verified on local cluster
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Re: Review Request 60671: AMBARI-21411. Backend - Run EU/RU PreChecks during a cross-stack upgrade

2017-07-05 Thread Alejandro Fernandez


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60671/#review179741
---




ambari-server/src/main/java/org/apache/ambari/server/controller/internal/PreUpgradeCheckResourceProvider.java
Lines 82 (patched)


New param must be passed in from the UI


- Alejandro Fernandez


On July 6, 2017, 12:21 a.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60671/
> ---
> 
> (Updated July 6, 2017, 12:21 a.m.)
> 
> 
> Review request for Ambari, Attila Doroszlai, Dmytro Grinenko, Jonathan 
> Hurley, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-21411
> https://issues.apache.org/jira/browse/AMBARI-21411
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> During a cross-stack upgrade, the PreChecks need to be able to run.
> 
> STR:
> 
> * Install BigInsights 4.2.5
> * Upgrade Ambari to 2.5
> * Register bits for HDP 2.6
> * Install bits for HDP 2.6
> * Attempt to run PreChecks
> 
> ```
> curl -u admin:admin -H "X-Requested-By: ambari" -i -X GET 
> http://localhost:8080/api/v1/clusters/c1/rolling_upgrades_check?fields=*&UpgradeChecks/repository_version=2.6.1.0-129&UpgradeChecks/upgrade_type=NON_ROLLING&UpgradeChecks/upgrade_pack=nonrolling-upgrade-to-hdp-2.6
> ```
> 
> This returns,
> ```
> {
>   "status" : 500,
>   "message" : "org.apache.ambari.server.controller.spi.SystemException: 
> Repository version null was not found"
> }
> ```
> 
> because in UpgradeHelper.java it cannot find the upgrade pack.
> 
> The new API will requiring passing in a new argument, e.g., 
> UpgradeChecks/target_stack=HDP-2.6
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/PreUpgradeCheckResourceProvider.java
>  7ccafb7 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java
>  795e8d1 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java 
> 69663fb 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/FLUME/metainfo.xml
>  99f1f79 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/JNBG/metainfo.xml
>  3520a32 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/metainfo.xml
>  4a7dc31 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/R4ML/metainfo.xml
>  d1c708d 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/metainfo.xml
>  be20810 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SYSTEMML/metainfo.xml
>  7a0e125 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/TITAN/metainfo.xml
>  d00e707 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/PreUpgradeCheckResourceProviderTest.java
>  47bba34 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/UpgradeHelperTest.java
>  6e27cba 
> 
> 
> Diff: https://reviews.apache.org/r/60671/diff/1/
> 
> 
> Testing
> ---
> 
> Verified on local cluster
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Review Request 60671: AMBARI-21411. Backend - Run EU/RU PreChecks during a cross-stack upgrade

2017-07-05 Thread Alejandro Fernandez


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60671/
---

Review request for Ambari, Attila Doroszlai, Dmytro Grinenko, Jonathan Hurley, 
Sumit Mohanty, and Sid Wagle.


Bugs: AMBARI-21411
https://issues.apache.org/jira/browse/AMBARI-21411


Repository: ambari


Description
---

During a cross-stack upgrade, the PreChecks need to be able to run.

STR:

* Install BigInsights 4.2.5
* Upgrade Ambari to 2.5
* Register bits for HDP 2.6
* Install bits for HDP 2.6
* Attempt to run PreChecks

```
curl -u admin:admin -H "X-Requested-By: ambari" -i -X GET 
http://localhost:8080/api/v1/clusters/c1/rolling_upgrades_check?fields=*&UpgradeChecks/repository_version=2.6.1.0-129&UpgradeChecks/upgrade_type=NON_ROLLING&UpgradeChecks/upgrade_pack=nonrolling-upgrade-to-hdp-2.6
```

This returns,
```
{
  "status" : 500,
  "message" : "org.apache.ambari.server.controller.spi.SystemException: 
Repository version null was not found"
}
```

because in UpgradeHelper.java it cannot find the upgrade pack.

The new API will requiring passing in a new argument, e.g., 
UpgradeChecks/target_stack=HDP-2.6


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/PreUpgradeCheckResourceProvider.java
 7ccafb7 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java
 795e8d1 
  ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java 
69663fb 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/FLUME/metainfo.xml
 99f1f79 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/JNBG/metainfo.xml
 3520a32 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/metainfo.xml
 4a7dc31 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/R4ML/metainfo.xml
 d1c708d 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SOLR/metainfo.xml
 be20810 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SYSTEMML/metainfo.xml
 7a0e125 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/TITAN/metainfo.xml
 d00e707 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/PreUpgradeCheckResourceProviderTest.java
 47bba34 
  
ambari-server/src/test/java/org/apache/ambari/server/state/UpgradeHelperTest.java
 6e27cba 


Diff: https://reviews.apache.org/r/60671/diff/1/


Testing
---

Verified on local cluster


Thanks,

Alejandro Fernandez

Re: Review Request 60604: AMBARI-21397. Remove previous upgrades to allow schema change

2017-07-05 Thread Jonathan Hurley



> On July 5, 2017, 4:15 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog252.java
> > Lines 121 (patched)
> > 
> >
> > Given that this is related to records in host_role_command, stage, 
> > request, we may need to delete some of those records as well and do a DB 
> > purge.

That was my comment from earlier. However, both Attila and I tested this out 
and there were no issues seen from just removing the upgrade records.


- Jonathan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60604/#review179697
---


On July 3, 2017, 2:05 p.m., Attila Doroszlai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60604/
> ---
> 
> (Updated July 3, 2017, 2:05 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, and Sumit 
> Mohanty.
> 
> 
> Bugs: AMBARI-21397
> https://issues.apache.org/jira/browse/AMBARI-21397
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Found references via foreign key constraints, added update/delete statements.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog252.java
>  fd26504481f68c7b85241547d4e823bc04912e4f 
> 
> 
> Diff: https://reviews.apache.org/r/60604/diff/1/
> 
> 
> Testing
> ---
> 
> Manual test on cluster that was upgraded prior to the upgrade to Ambari 2.5.2.
> 
> 
> Thanks,
> 
> Attila Doroszlai
> 
>

Re: Review Request 60389: Extract Log Search part from stack advisor to a separate service advisor

2017-07-05 Thread Alejandro Fernandez


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60389/#review179704
---


Fix it, then Ship it!





ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/service_advisor.py
Lines 114 (patched)


Add documentation for each one of the recommendations here.


- Alejandro Fernandez


On June 30, 2017, 10:34 a.m., Miklos Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60389/
> ---
> 
> (Updated June 30, 2017, 10:34 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Oliver Szabo, and Robert 
> Nettleton.
> 
> 
> Bugs: AMBARI-21333
> https://issues.apache.org/jira/browse/AMBARI-21333
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Extract Log Search service advisor from the stack advisor.
> Also created an empty Ambari Infra service advisor.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/service_advisor.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/service_advisor.py
>  PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.2/services/stack_advisor.py 
> 726514b 
>   
> ambari-server/src/test/python/common-services/LOGSEARCH/test_service_advisor.py
>  PRE-CREATION 
>   ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 
> c28ee2a 
> 
> 
> Diff: https://reviews.apache.org/r/60389/diff/3/
> 
> 
> Testing
> ---
> 
> Tested locally.
> 
> 
> Thanks,
> 
> Miklos Gergely
> 
>

Re: Review Request 60655: AMBARI-21405. Create custom action to force-remove packages

2017-07-05 Thread Attila Doroszlai



> On júl. 5, 2017, 8:11 du, Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/custom_actions/scripts/force_remove_packages.py
> > Lines 41 (patched)
> > 
> >
> > Is this initialized to an empty list?

Yes, it is.


- Attila


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60655/#review179669
---


On júl. 5, 2017, 2:56 du, Attila Doroszlai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60655/
> ---
> 
> (Updated júl. 5, 2017, 2:56 du)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Sumit 
> Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-21405
> https://issues.apache.org/jira/browse/AMBARI-21405
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> * Add option in `Package` resource to remove package while leaving its 
> dependencies installed
> * Add custom action `force_remove_packages`
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/test/python/resource_management/TestPackageResource.py 
> 66227c66a7768aa044f30a1c76d782c5101be678 
>   
> ambari-common/src/main/python/resource_management/core/providers/package/__init__.py
>  21de1839c853794f252a4cb2265b89549a802d88 
>   
> ambari-common/src/main/python/resource_management/core/providers/package/apt.py
>  d095173f07c46842b6e9e38dc933e06a3e43a749 
>   
> ambari-common/src/main/python/resource_management/core/providers/package/choco.py
>  db552964279f509de90d2fde884bfc0fea12f7b0 
>   
> ambari-common/src/main/python/resource_management/core/providers/package/yumrpm.py
>  ea10a8601430b62bbe216848e1d26a5e8bc16927 
>   
> ambari-common/src/main/python/resource_management/core/providers/package/zypper.py
>  265c1626873bfb98196f102e73bb2dbedee94c80 
>   
> ambari-common/src/main/python/resource_management/core/resources/packaging.py 
> e3adc30f46abd3f33670f817421d88270b3e7ade 
>   
> ambari-server/src/main/resources/custom_action_definitions/system_action_definitions.xml
>  0f502568ddfb5da7d5574b2e33c6a354c37d9697 
>   
> ambari-server/src/main/resources/custom_actions/scripts/force_remove_packages.py
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/60655/diff/1/
> 
> 
> Testing
> ---
> 
> Tested custom action as part of cross-stack upgrade on CentOS7.
> 
> Manually tested custom action on Debian 7.6:
> 
> ```
> $ curl -X POST -d '{ "RequestInfo": { "action": "force_remove_packages", 
> "parameters": { "package_list": "vim-common" } } }' 
> http://localhost:8080/api/v1/clusters/TEST/requests
> 
> output.txt:
> 2017-07-05 12:29:59,400 - Package['vim-common'] {'action': ['remove'], 
> 'remove_with_dependencies': False}
> 2017-07-05 12:29:59,411 - Removing package vim-common ('/usr/bin/dpkg 
> --remove --ignore-depends vim-common vim-common')
> 2017-07-05 12:30:00,702 - Removed vim-common
> 
> $ apt-get install vim
> ...
> vim is already the newest version.
> You might want to run 'apt-get -f install' to correct these:
> The following packages have unmet dependencies:
>  vim : Depends: vim-common (= 2:7.3.547-7+deb7u3) but it is not going to be 
> installed
> ```
> 
> New unit tests for `Package` in `ambari-agent`:
> 
> ```
> Ran 456 tests in 38.256s
> 
> OK
> ```
> 
> 
> Thanks,
> 
> Attila Doroszlai
> 
>

Review Request 60668: Fixed outdated index.html generation in ambari-server/target/generated-sources/swagger

2017-07-05 Thread Balázs Bence Sári


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60668/
---

Review request for Ambari, Attila Doroszlai, Jaimin Jetly, Oliver Szabo, and 
Sebastian Toader.


Bugs: AMBARI-21410
https://issues.apache.org/jira/browse/AMBARI-21410


Repository: ambari


Description
---

The input for html generation is taken from the correct source 
(ambari-server/target/generated-sources/swagger/swagger.json instead of 
ambari-server/docs/api/generated/swagger.json).


Diffs
-

  ambari-server/pom.xml 878665e 


Diff: https://reviews.apache.org/r/60668/diff/1/


Testing
---

Manually tested docs generation both with and without the 
-Dgenerate.swagger.resources flag.


Thanks,

Balázs Bence Sári

Re: Review Request 60604: AMBARI-21397. Remove previous upgrades to allow schema change

2017-07-05 Thread Alejandro Fernandez


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60604/#review179697
---




ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog252.java
Lines 121 (patched)


Given that this is related to records in host_role_command, stage, request, 
we may need to delete some of those records as well and do a DB purge.


- Alejandro Fernandez


On July 3, 2017, 6:05 p.m., Attila Doroszlai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60604/
> ---
> 
> (Updated July 3, 2017, 6:05 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, and Sumit 
> Mohanty.
> 
> 
> Bugs: AMBARI-21397
> https://issues.apache.org/jira/browse/AMBARI-21397
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Found references via foreign key constraints, added update/delete statements.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog252.java
>  fd26504481f68c7b85241547d4e823bc04912e4f 
> 
> 
> Diff: https://reviews.apache.org/r/60604/diff/1/
> 
> 
> Testing
> ---
> 
> Manual test on cluster that was upgraded prior to the upgrade to Ambari 2.5.2.
> 
> 
> Thanks,
> 
> Attila Doroszlai
> 
>

Re: Review Request 60560: AMBARI-21374 : Replication factor should be configurable in Atlas to create solr collections

2017-07-05 Thread Alejandro Fernandez


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60560/#review179696
---


Ship it!




Ship It!

- Alejandro Fernandez


On July 3, 2017, 5:35 a.m., Vishal Suvagia wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60560/
> ---
> 
> (Updated July 3, 2017, 5:35 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Gautam Borad, Jayush Luniya, 
> Madhan Neethiraj, Mugdha Varadkar, Nixon Rodrigues, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-21374
> https://issues.apache.org/jira/browse/AMBARI-21374
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In current scenarios, Solr collections created for Atlas, has 
> replication-factor dependent on the no of Infra-Solr servers present in 
> Ambari, However to be more modular we should allow the replication factor to 
> be configurable for the user.
> PS: Attached patch is for trunk branch, patch for branch-2.5 is attached to 
> Apache JIRA.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py
>  111a248 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.7.0.3.0/configuration/atlas-env.xml
>  c5a4fd6 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.7.0.3.0/package/scripts/params.py
>  d26df33 
>   
> ambari-server/src/main/resources/stacks/HDP/2.6/services/ATLAS/configuration/atlas-env.xml
>  d54cf15 
> 
> 
> Diff: https://reviews.apache.org/r/60560/diff/2/
> 
> 
> Testing
> ---
> 
> Verified with installation on Cent-OS 6.
> 
> 
> Thanks,
> 
> Vishal Suvagia
> 
>

Review Request 60665: Add swagger annotations to the new AmbariConfigurationService class.

2017-07-05 Thread Balázs Bence Sári


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60665/
---

Review request for Ambari, Attila Doroszlai, Laszlo Puskas, Oliver Szabo, and 
Sebastian Toader.


Bugs: AMBARI-21307
https://issues.apache.org/jira/browse/AMBARI-21307


Repository: ambari


Description
---

Added swagger annotations to the new AmbariConfigurationService class.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariConfigurationRequestSwagger.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariConfigurationResponseSwagger.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariConfigurationService.java
 0fa6e44 


Diff: https://reviews.apache.org/r/60665/diff/1/


Testing
---

Manually checked the generated html.


Thanks,

Balázs Bence Sári

Re: Review Request 60560: AMBARI-21374 : Replication factor should be configurable in Atlas to create solr collections

2017-07-05 Thread Sumit Mohanty


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60560/#review179483
---


Ship it!




Ship It!

- Sumit Mohanty


On July 3, 2017, 5:35 a.m., Vishal Suvagia wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60560/
> ---
> 
> (Updated July 3, 2017, 5:35 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Gautam Borad, Jayush Luniya, 
> Madhan Neethiraj, Mugdha Varadkar, Nixon Rodrigues, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-21374
> https://issues.apache.org/jira/browse/AMBARI-21374
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In current scenarios, Solr collections created for Atlas, has 
> replication-factor dependent on the no of Infra-Solr servers present in 
> Ambari, However to be more modular we should allow the replication factor to 
> be configurable for the user.
> PS: Attached patch is for trunk branch, patch for branch-2.5 is attached to 
> Apache JIRA.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py
>  111a248 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.7.0.3.0/configuration/atlas-env.xml
>  c5a4fd6 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.7.0.3.0/package/scripts/params.py
>  d26df33 
>   
> ambari-server/src/main/resources/stacks/HDP/2.6/services/ATLAS/configuration/atlas-env.xml
>  d54cf15 
> 
> 
> Diff: https://reviews.apache.org/r/60560/diff/2/
> 
> 
> Testing
> ---
> 
> Verified with installation on Cent-OS 6.
> 
> 
> Thanks,
> 
> Vishal Suvagia
> 
>

Re: Review Request 60403: Add Kerberos HTTP SPNEGO authentication support to Ambari Metrics Monitor

2017-07-05 Thread Robert Levas


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60403/#review179673
---


Fix it, then Ship it!




Ship It!


ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/spnego_kerberos_auth.py
Lines 33 (patched)


Would it be possible to use 
`resource_management.libraries.functions.curl_krb_request.curl_krb_request` 
instead?


- Robert Levas


On June 23, 2017, 6:49 p.m., Qin Liu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60403/
> ---
> 
> (Updated June 23, 2017, 6:49 p.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Di Li, Dmytro Sen, Robert 
> Levas, Sid Wagle, and Tim Thorpe.
> 
> 
> Bugs: AMBARI-17898
> https://issues.apache.org/jira/browse/AMBARI-17898
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> This is a subtask of AMBARI-14384 "Ambari Metrics doesn't use SPNEGO to 
> authenticate".
> In a Kerberos enabled cluster with SPNEGO enabled on Hadoop APIs, Ambari 
> Metrics Collector web-console will be Kerberos HTTP SPNEGO enabled too. But 
> Ambari Metrics Monitor, a client of Ambari Metrics Collector, currently does 
> not support Kerberos HTTP SPNEGO authentication.
> /var/log/ambari-metrics-monitor/ambari-metrics-monitor.out:
> 2015-12-15 13:26:30,663 [INFO] emitter.py:101 - server: 
> http://metrics-collector:6188/ws/v1/timeline/metrics
> 2015-12-15 13:26:30,671 [WARNING] emitter.py:84 - Error sending metrics to 
> server. HTTP Error 401: Authentication required
> 2015-12-15 13:26:30,671 [WARNING] emitter.py:90 - Retrying after 5 ...
> 
> 
> Diffs
> -
> 
>   ambari-metrics/ambari-metrics-host-monitoring/conf/unix/metric_monitor.ini 
> 7fe7397 
>   
> ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/config_reader.py
>  d1429ed 
>   
> ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/emitter.py 
> 77b8c23 
>   
> ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/krberr.py 
> PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/spnego_kerberos_auth.py
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-host-monitoring/src/test/python/core/TestEmitter.py
>  4056ae3 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/kerberos.json
>  51f541f 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/metainfo.xml
>  9031b46 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/ams.py
>  f49d47d 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py
>  b8c14f4 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/templates/metric_monitor.ini.j2
>  b7dee50 
> 
> 
> Diff: https://reviews.apache.org/r/60403/diff/1/
> 
> 
> Testing
> ---
> 
> 1. Unit tests passed
> 2. manually tested with HDP/trunk
> 
> 
> Thanks,
> 
> Qin Liu
> 
>

Re: Review Request 60249: AMBARI-21234 Ambari rack awareness for Kafka

2017-07-05 Thread Ambud Sharma



> On July 5, 2017, 6:12 p.m., Sriharsha Chintalapani wrote:
> > ambari-server/src/main/resources/common-services/KAFKA/0.8.1/package/scripts/kafka.py
> > Lines 117 (patched)
> > 
> >
> > how are we co-relating the host and the all_racks array here?
> > Are we expecting the hosts to be sorted and all_racks to match with 
> > all_racks index to pick the config.
> > This might break if the user later adds another broker.

Yes, the expectation is that array indices are the same. This approach is what 
HDFS stack uses as well: 
https://github.com/apache/ambari/blob/79cca1c7184f1661236971dac70d85a83fab6c11/ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/templates/topology_mappings.data.j2


> On July 5, 2017, 6:12 p.m., Sriharsha Chintalapani wrote:
> > ambari-server/src/main/resources/common-services/KAFKA/0.8.1/package/scripts/kafka.py
> > Lines 121 (patched)
> > 
> >
> > This will add the broker.rack to all the versions. Can we make sure we 
> > only add this 0.10.x onwards

Yes, we are trying to see if the 0.10.x stack from Ambari trunk can be 
backported to branch-2.5, this will limit the scope of the rack configuration 
to only 0.10.x onwards.


- Ambud


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60249/#review179667
---


On July 5, 2017, 6:14 p.m., Ambud Sharma wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60249/
> ---
> 
> (Updated July 5, 2017, 6:14 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez and Sriharsha Chintalapani.
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Ambari rack awareness for Kafka. 
> https://issues.apache.org/jira/browse/AMBARI-21234
> 
> As an operations person it would be nice to manage Kafka rack awareness via 
> Ambari. Ambari allows node rack information to be configured and this 
> information can then be pulled in the Kafka stack and populated in the 
> server.properties file for Kafka.
> Design:
> This stack change uses the /clusterHostInfo/all_hosts and 
> /clusterHostInfo/all_racks paths and materializes them to a variable. Then it 
> uses linear search to find this node in the list of all hosts and it's 
> corresponding rack id. This information is then stored in a variable called 
> rack and which is materialized during the configure method of the broker 
> scripts.
> This stack change relies on the node rack information stored in Ambari 
> therefore will enable both Ambari UI and Blueprints to be used for setting up 
> Kafka broker rack information.
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/common-services/KAFKA/0.8.1/metainfo.xml 
> d322adc47b 
>   
> ambari-server/src/main/resources/common-services/KAFKA/0.8.1/package/scripts/kafka.py
>  1327090aa7 
>   
> ambari-server/src/main/resources/common-services/KAFKA/0.8.1/package/scripts/params.py
>  c36a10ff28 
> 
> 
> Diff: https://reviews.apache.org/r/60249/diff/5/
> 
> 
> Testing
> ---
> 
> Manually deployed Kafka cluster and verified the broker.rack property is 
> correctly populated.
> 
> 
> Thanks,
> 
> Ambud Sharma
> 
>

Re: Review Request 60249: AMBARI-21234 Ambari rack awareness for Kafka

2017-07-05 Thread Sriharsha Chintalapani


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60249/#review179667
---




ambari-server/src/main/resources/common-services/KAFKA/0.8.1/package/scripts/kafka.py
Lines 117 (patched)


how are we co-relating the host and the all_racks array here?
Are we expecting the hosts to be sorted and all_racks to match with 
all_racks index to pick the config.
This might break if the user later adds another broker.



ambari-server/src/main/resources/common-services/KAFKA/0.8.1/package/scripts/kafka.py
Lines 121 (patched)


This will add the broker.rack to all the versions. Can we make sure we only 
add this 0.10.x onwards


- Sriharsha Chintalapani


On July 4, 2017, 6:53 p.m., Ambud Sharma wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60249/
> ---
> 
> (Updated July 4, 2017, 6:53 p.m.)
> 
> 
> Review request for Ambari and Alejandro Fernandez.
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Ambari rack awareness for Kafka. 
> https://issues.apache.org/jira/browse/AMBARI-21234
> 
> As an operations person it would be nice to manage Kafka rack awareness via 
> Ambari. Ambari allows node rack information to be configured and this 
> information can then be pulled in the Kafka stack and populated in the 
> server.properties file for Kafka.
> Design:
> This stack change uses the /clusterHostInfo/all_hosts and 
> /clusterHostInfo/all_racks paths and materializes them to a variable. Then it 
> uses linear search to find this node in the list of all hosts and it's 
> corresponding rack id. This information is then stored in a variable called 
> rack and which is materialized during the configure method of the broker 
> scripts.
> This stack change relies on the node rack information stored in Ambari 
> therefore will enable both Ambari UI and Blueprints to be used for setting up 
> Kafka broker rack information.
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/common-services/KAFKA/0.8.1/metainfo.xml 
> d322adc47b 
>   
> ambari-server/src/main/resources/common-services/KAFKA/0.8.1/package/scripts/kafka.py
>  1327090aa7 
>   
> ambari-server/src/main/resources/common-services/KAFKA/0.8.1/package/scripts/params.py
>  c36a10ff28 
> 
> 
> Diff: https://reviews.apache.org/r/60249/diff/5/
> 
> 
> Testing
> ---
> 
> Manually deployed Kafka cluster and verified the broker.rack property is 
> correctly populated.
> 
> 
> Thanks,
> 
> Ambud Sharma
> 
>

Re: Review Request 60655: AMBARI-21405. Create custom action to force-remove packages

2017-07-05 Thread Alejandro Fernandez


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60655/#review179669
---




ambari-server/src/main/resources/custom_actions/scripts/force_remove_packages.py
Lines 41 (patched)


Is this initialized to an empty list?


- Alejandro Fernandez


On July 5, 2017, 12:56 p.m., Attila Doroszlai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60655/
> ---
> 
> (Updated July 5, 2017, 12:56 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Sumit 
> Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-21405
> https://issues.apache.org/jira/browse/AMBARI-21405
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> * Add option in `Package` resource to remove package while leaving its 
> dependencies installed
> * Add custom action `force_remove_packages`
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/test/python/resource_management/TestPackageResource.py 
> 66227c66a7768aa044f30a1c76d782c5101be678 
>   
> ambari-common/src/main/python/resource_management/core/providers/package/__init__.py
>  21de1839c853794f252a4cb2265b89549a802d88 
>   
> ambari-common/src/main/python/resource_management/core/providers/package/apt.py
>  d095173f07c46842b6e9e38dc933e06a3e43a749 
>   
> ambari-common/src/main/python/resource_management/core/providers/package/choco.py
>  db552964279f509de90d2fde884bfc0fea12f7b0 
>   
> ambari-common/src/main/python/resource_management/core/providers/package/yumrpm.py
>  ea10a8601430b62bbe216848e1d26a5e8bc16927 
>   
> ambari-common/src/main/python/resource_management/core/providers/package/zypper.py
>  265c1626873bfb98196f102e73bb2dbedee94c80 
>   
> ambari-common/src/main/python/resource_management/core/resources/packaging.py 
> e3adc30f46abd3f33670f817421d88270b3e7ade 
>   
> ambari-server/src/main/resources/custom_action_definitions/system_action_definitions.xml
>  0f502568ddfb5da7d5574b2e33c6a354c37d9697 
>   
> ambari-server/src/main/resources/custom_actions/scripts/force_remove_packages.py
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/60655/diff/1/
> 
> 
> Testing
> ---
> 
> Tested custom action as part of cross-stack upgrade on CentOS7.
> 
> Manually tested custom action on Debian 7.6:
> 
> ```
> $ curl -X POST -d '{ "RequestInfo": { "action": "force_remove_packages", 
> "parameters": { "package_list": "vim-common" } } }' 
> http://localhost:8080/api/v1/clusters/TEST/requests
> 
> output.txt:
> 2017-07-05 12:29:59,400 - Package['vim-common'] {'action': ['remove'], 
> 'remove_with_dependencies': False}
> 2017-07-05 12:29:59,411 - Removing package vim-common ('/usr/bin/dpkg 
> --remove --ignore-depends vim-common vim-common')
> 2017-07-05 12:30:00,702 - Removed vim-common
> 
> $ apt-get install vim
> ...
> vim is already the newest version.
> You might want to run 'apt-get -f install' to correct these:
> The following packages have unmet dependencies:
>  vim : Depends: vim-common (= 2:7.3.547-7+deb7u3) but it is not going to be 
> installed
> ```
> 
> New unit tests for `Package` in `ambari-agent`:
> 
> ```
> Ran 456 tests in 38.256s
> 
> OK
> ```
> 
> 
> Thanks,
> 
> Attila Doroszlai
> 
>

Re: Review Request 60655: AMBARI-21405. Create custom action to force-remove packages

2017-07-05 Thread Jonathan Hurley


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60655/#review179664
---




ambari-server/src/main/resources/custom_action_definitions/system_action_definitions.xml
Lines 104 (patched)


Change to: Remove packages specified by package_list, instructing the 
package manager to ignore dependencies.



ambari-server/src/main/resources/custom_actions/scripts/force_remove_packages.py
Lines 46 (patched)


Can you specify which packages failed here? Or, at least, the list of 
packages attempted?


- Jonathan Hurley


On July 5, 2017, 8:56 a.m., Attila Doroszlai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60655/
> ---
> 
> (Updated July 5, 2017, 8:56 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Sumit 
> Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-21405
> https://issues.apache.org/jira/browse/AMBARI-21405
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> * Add option in `Package` resource to remove package while leaving its 
> dependencies installed
> * Add custom action `force_remove_packages`
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/test/python/resource_management/TestPackageResource.py 
> 66227c66a7768aa044f30a1c76d782c5101be678 
>   
> ambari-common/src/main/python/resource_management/core/providers/package/__init__.py
>  21de1839c853794f252a4cb2265b89549a802d88 
>   
> ambari-common/src/main/python/resource_management/core/providers/package/apt.py
>  d095173f07c46842b6e9e38dc933e06a3e43a749 
>   
> ambari-common/src/main/python/resource_management/core/providers/package/choco.py
>  db552964279f509de90d2fde884bfc0fea12f7b0 
>   
> ambari-common/src/main/python/resource_management/core/providers/package/yumrpm.py
>  ea10a8601430b62bbe216848e1d26a5e8bc16927 
>   
> ambari-common/src/main/python/resource_management/core/providers/package/zypper.py
>  265c1626873bfb98196f102e73bb2dbedee94c80 
>   
> ambari-common/src/main/python/resource_management/core/resources/packaging.py 
> e3adc30f46abd3f33670f817421d88270b3e7ade 
>   
> ambari-server/src/main/resources/custom_action_definitions/system_action_definitions.xml
>  0f502568ddfb5da7d5574b2e33c6a354c37d9697 
>   
> ambari-server/src/main/resources/custom_actions/scripts/force_remove_packages.py
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/60655/diff/1/
> 
> 
> Testing
> ---
> 
> Tested custom action as part of cross-stack upgrade on CentOS7.
> 
> Manually tested custom action on Debian 7.6:
> 
> ```
> $ curl -X POST -d '{ "RequestInfo": { "action": "force_remove_packages", 
> "parameters": { "package_list": "vim-common" } } }' 
> http://localhost:8080/api/v1/clusters/TEST/requests
> 
> output.txt:
> 2017-07-05 12:29:59,400 - Package['vim-common'] {'action': ['remove'], 
> 'remove_with_dependencies': False}
> 2017-07-05 12:29:59,411 - Removing package vim-common ('/usr/bin/dpkg 
> --remove --ignore-depends vim-common vim-common')
> 2017-07-05 12:30:00,702 - Removed vim-common
> 
> $ apt-get install vim
> ...
> vim is already the newest version.
> You might want to run 'apt-get -f install' to correct these:
> The following packages have unmet dependencies:
>  vim : Depends: vim-common (= 2:7.3.547-7+deb7u3) but it is not going to be 
> installed
> ```
> 
> New unit tests for `Package` in `ambari-agent`:
> 
> ```
> Ran 456 tests in 38.256s
> 
> OK
> ```
> 
> 
> Thanks,
> 
> Attila Doroszlai
> 
>

Re: Review Request 60655: AMBARI-21405. Create custom action to force-remove packages

2017-07-05 Thread Jonathan Hurley


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60655/#review179663
---




ambari-common/src/main/python/resource_management/core/providers/package/apt.py
Line 106 (original), 107 (patched)


For clarity, can we change this naming convention from "with" to "ignore". 
Using "with" makes it seem like you're also removing dependencies. A simple 
ignore_dependencies=False would be good.


- Jonathan Hurley


On July 5, 2017, 8:56 a.m., Attila Doroszlai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60655/
> ---
> 
> (Updated July 5, 2017, 8:56 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Sumit 
> Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-21405
> https://issues.apache.org/jira/browse/AMBARI-21405
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> * Add option in `Package` resource to remove package while leaving its 
> dependencies installed
> * Add custom action `force_remove_packages`
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/test/python/resource_management/TestPackageResource.py 
> 66227c66a7768aa044f30a1c76d782c5101be678 
>   
> ambari-common/src/main/python/resource_management/core/providers/package/__init__.py
>  21de1839c853794f252a4cb2265b89549a802d88 
>   
> ambari-common/src/main/python/resource_management/core/providers/package/apt.py
>  d095173f07c46842b6e9e38dc933e06a3e43a749 
>   
> ambari-common/src/main/python/resource_management/core/providers/package/choco.py
>  db552964279f509de90d2fde884bfc0fea12f7b0 
>   
> ambari-common/src/main/python/resource_management/core/providers/package/yumrpm.py
>  ea10a8601430b62bbe216848e1d26a5e8bc16927 
>   
> ambari-common/src/main/python/resource_management/core/providers/package/zypper.py
>  265c1626873bfb98196f102e73bb2dbedee94c80 
>   
> ambari-common/src/main/python/resource_management/core/resources/packaging.py 
> e3adc30f46abd3f33670f817421d88270b3e7ade 
>   
> ambari-server/src/main/resources/custom_action_definitions/system_action_definitions.xml
>  0f502568ddfb5da7d5574b2e33c6a354c37d9697 
>   
> ambari-server/src/main/resources/custom_actions/scripts/force_remove_packages.py
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/60655/diff/1/
> 
> 
> Testing
> ---
> 
> Tested custom action as part of cross-stack upgrade on CentOS7.
> 
> Manually tested custom action on Debian 7.6:
> 
> ```
> $ curl -X POST -d '{ "RequestInfo": { "action": "force_remove_packages", 
> "parameters": { "package_list": "vim-common" } } }' 
> http://localhost:8080/api/v1/clusters/TEST/requests
> 
> output.txt:
> 2017-07-05 12:29:59,400 - Package['vim-common'] {'action': ['remove'], 
> 'remove_with_dependencies': False}
> 2017-07-05 12:29:59,411 - Removing package vim-common ('/usr/bin/dpkg 
> --remove --ignore-depends vim-common vim-common')
> 2017-07-05 12:30:00,702 - Removed vim-common
> 
> $ apt-get install vim
> ...
> vim is already the newest version.
> You might want to run 'apt-get -f install' to correct these:
> The following packages have unmet dependencies:
>  vim : Depends: vim-common (= 2:7.3.547-7+deb7u3) but it is not going to be 
> installed
> ```
> 
> New unit tests for `Package` in `ambari-agent`:
> 
> ```
> Ran 456 tests in 38.256s
> 
> OK
> ```
> 
> 
> Thanks,
> 
> Attila Doroszlai
> 
>

Re: Review Request 60656: Ambari agent not starting in latest M17 PPC build

2017-07-05 Thread Sumit Mohanty


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60656/#review179658
---


Ship it!




Ship It!

- Sumit Mohanty


On July 5, 2017, 5:20 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60656/
> ---
> 
> (Updated July 5, 2017, 5:20 p.m.)
> 
> 
> Review request for Ambari and Dmitro Lisnichenko.
> 
> 
> Bugs: AMBARI-21409
> https://issues.apache.org/jira/browse/AMBARI-21409
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Build:
> 
> 
> 
> 
> 
> http://s3.amazonaws.com/dev.hortonworks.com/ambari/centos7-ppc/2.x/BUILDS/2.5.2.0-95
> 
> 
> 
> 
> INFO 2017-07-03 08:53:46,092 NetUtil.py:70 - Connecting to 
> https://nat-r7-kwtu-flumeng-5.openstacklocal:8440/connection_info
> INFO 2017-07-03 08:53:46,167 security.py:93 - SSL Connect being called.. 
> connecting to the server
> INFO 2017-07-03 08:53:46,257 security.py:60 - SSL connection established. 
> Two-way SSL authentication is turned off on the server.
> ERROR 2017-07-03 08:53:46,266 Controller.py:190 - Cannot register host 
> with not supported os type, hostname=nat-r7-kwtu-flumeng-5.openstacklocal, 
> serverOsType=redhat-ppc7, agentOsType=redhat-ppc7
> INFO 2017-07-03 08:53:46,266 Controller.py:516 - Registration response 
> from nat-r7-kwtu-flumeng-5.openstacklocal was FAILED
> INFO 2017-07-03 08:53:46,266 Controller.py:531 - Registration response 
> from %s didn't contain 'response' as a key
> INFO 2017-07-03 08:53:46,266 Controller.py:503 - Finished heartbeating 
> and registering cycle
> INFO 2017-07-03 08:53:46,266 Controller.py:509 - Controller thread has 
> successfully finished
> INFO 2017-07-03 08:53:47,959 ExitHelper.py:56 - Performing cleanup before 
> exiting...
> 
> 
> Diffs
> -
> 
>   ambari-common/src/main/python/ambari_commons/resources/os_family.json 
> b9cdbf6 
> 
> 
> Diff: https://reviews.apache.org/r/60656/diff/1/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Review Request 60656: Ambari agent not starting in latest M17 PPC build

2017-07-05 Thread Andrew Onischuk


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60656/
---

Review request for Ambari and Dmitro Lisnichenko.


Bugs: AMBARI-21409
https://issues.apache.org/jira/browse/AMBARI-21409


Repository: ambari


Description
---

Build:





http://s3.amazonaws.com/dev.hortonworks.com/ambari/centos7-ppc/2.x/BUILDS/2.5.2.0-95




INFO 2017-07-03 08:53:46,092 NetUtil.py:70 - Connecting to 
https://nat-r7-kwtu-flumeng-5.openstacklocal:8440/connection_info
INFO 2017-07-03 08:53:46,167 security.py:93 - SSL Connect being called.. 
connecting to the server
INFO 2017-07-03 08:53:46,257 security.py:60 - SSL connection established. 
Two-way SSL authentication is turned off on the server.
ERROR 2017-07-03 08:53:46,266 Controller.py:190 - Cannot register host with 
not supported os type, hostname=nat-r7-kwtu-flumeng-5.openstacklocal, 
serverOsType=redhat-ppc7, agentOsType=redhat-ppc7
INFO 2017-07-03 08:53:46,266 Controller.py:516 - Registration response from 
nat-r7-kwtu-flumeng-5.openstacklocal was FAILED
INFO 2017-07-03 08:53:46,266 Controller.py:531 - Registration response from 
%s didn't contain 'response' as a key
INFO 2017-07-03 08:53:46,266 Controller.py:503 - Finished heartbeating and 
registering cycle
INFO 2017-07-03 08:53:46,266 Controller.py:509 - Controller thread has 
successfully finished
INFO 2017-07-03 08:53:47,959 ExitHelper.py:56 - Performing cleanup before 
exiting...


Diffs
-

  ambari-common/src/main/python/ambari_commons/resources/os_family.json b9cdbf6 


Diff: https://reviews.apache.org/r/60656/diff/1/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Re: Review Request 60595: AMBARI-21390. Change stack root references in config during cross-stack upgrade

2017-07-05 Thread Alejandro Fernandez



> On July 5, 2017, 5:09 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/ChangeStackRootDirectoryAction.java
> > Lines 74 (patched)
> > 
> >
> > We need to change other things just besides the stack root.
> > There are other references to 
> > 
> > iop/apps
> > iop-select
> > 
> > Because this is specific to IOP to HDP, I suggest putting IOPtoHDP in 
> > the class name.

Also things like $iop.version


- Alejandro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60595/#review179655
---


On July 3, 2017, 2:19 p.m., Attila Doroszlai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60595/
> ---
> 
> (Updated July 3, 2017, 2:19 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, and Sumit 
> Mohanty.
> 
> 
> Bugs: AMBARI-21390
> https://issues.apache.org/jira/browse/AMBARI-21390
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Old and new stack root paths are hard-coded for now.  I will try to make it 
> configurable later.
> The patch also copies BigInsights 4.2 upgrade pack to 4.2.5 to be able to 
> upgrade from the latter version.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/ChangeStackRootDirectoryAction.java
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/config-upgrade.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
>  e8a1e33545c17faaf5856b77b77c4621249bb4ac 
>   
> ambari-server/src/test/java/org/apache/ambari/server/serveraction/upgrades/ChangeStackRootDirectoryActionTest.java
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/60595/diff/2/
> 
> 
> Testing
> ---
> 
> Manual test on local cluster.
> New unit test.
> 
> 
> Thanks,
> 
> Attila Doroszlai
> 
>

Re: Review Request 60595: AMBARI-21390. Change stack root references in config during cross-stack upgrade

2017-07-05 Thread Alejandro Fernandez


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60595/#review179655
---




ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/ChangeStackRootDirectoryAction.java
Lines 74 (patched)


We need to change other things just besides the stack root.
There are other references to 

iop/apps
iop-select

Because this is specific to IOP to HDP, I suggest putting IOPtoHDP in the 
class name.


- Alejandro Fernandez


On July 3, 2017, 2:19 p.m., Attila Doroszlai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60595/
> ---
> 
> (Updated July 3, 2017, 2:19 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, and Sumit 
> Mohanty.
> 
> 
> Bugs: AMBARI-21390
> https://issues.apache.org/jira/browse/AMBARI-21390
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Old and new stack root paths are hard-coded for now.  I will try to make it 
> configurable later.
> The patch also copies BigInsights 4.2 upgrade pack to 4.2.5 to be able to 
> upgrade from the latter version.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/ChangeStackRootDirectoryAction.java
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/config-upgrade.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
>  e8a1e33545c17faaf5856b77b77c4621249bb4ac 
>   
> ambari-server/src/test/java/org/apache/ambari/server/serveraction/upgrades/ChangeStackRootDirectoryActionTest.java
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/60595/diff/2/
> 
> 
> Testing
> ---
> 
> Manual test on local cluster.
> New unit test.
> 
> 
> Thanks,
> 
> Attila Doroszlai
> 
>

Re: Review Request 60460: AMBARI-21058 HDP 3.0 - Changing common service version for Ranger & Ranger Kms

2017-07-05 Thread Alejandro Fernandez


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60460/#review179654
---


Ship it!




Ship It!

- Alejandro Fernandez


On June 27, 2017, 10:34 a.m., Mugdha Varadkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60460/
> ---
> 
> (Updated June 27, 2017, 10:34 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Gautam Borad, and Jayush 
> Luniya.
> 
> 
> Bugs: AMBARI-21058
> https://issues.apache.org/jira/browse/AMBARI-21058
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> 1) Changing of common service version to be 1.0.0.3.0
> 2) Some modifications to Python scripts.
> 3) Merging all theme file related changes in one theme file for Ranger and 
> Ranger Kms
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/service_advisor.py
>  356ad59 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.7.0.3.0/alerts.json 
> ab473a8 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.7.0.3.0/configuration/admin-log4j.xml
>  fbbfac7 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.7.0.3.0/configuration/admin-properties.xml
>  1d73087 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.7.0.3.0/configuration/atlas-tagsync-ssl.xml
>  d43c010 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.7.0.3.0/configuration/ranger-admin-site.xml
>  a9153f8 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.7.0.3.0/configuration/ranger-env.xml
>  3e25470 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.7.0.3.0/configuration/ranger-site.xml
>  c70e222 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.7.0.3.0/configuration/ranger-solr-configuration.xml
>  550ce0d 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.7.0.3.0/configuration/ranger-tagsync-policymgr-ssl.xml
>  a4c9441 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.7.0.3.0/configuration/ranger-tagsync-site.xml
>  5e60c06 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.7.0.3.0/configuration/ranger-ugsync-site.xml
>  922fbd8 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.7.0.3.0/configuration/tagsync-application-properties.xml
>  f616324 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.7.0.3.0/configuration/tagsync-log4j.xml
>  8ec85a0 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.7.0.3.0/configuration/usersync-log4j.xml
>  6d91b6e 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.7.0.3.0/configuration/usersync-properties.xml
>  15aabe8 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.7.0.3.0/kerberos.json
>  1fc8acf 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.7.0.3.0/metainfo.xml
>  e208800 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.7.0.3.0/package/alerts/alert_ranger_admin_passwd_check.py
>  8ea8070 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.7.0.3.0/package/scripts/params.py
>  094d239 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.7.0.3.0/package/scripts/ranger_admin.py
>  b849d58 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.7.0.3.0/package/scripts/ranger_service.py
>  0355049 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.7.0.3.0/package/scripts/ranger_tagsync.py
>  008fb99 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.7.0.3.0/package/scripts/ranger_usersync.py
>  b9366f6 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.7.0.3.0/package/scripts/service_check.py
>  fb6af95 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.7.0.3.0/package/scripts/setup_ranger.py
>  b0e8bad 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.7.0.3.0/package/scripts/setup_ranger_xml.py
>  26e6578 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.7.0.3.0/package/scripts/status_params.py
>  842430b 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.7.0.3.0/package/scripts/upgrade.py
>  a07a1fd 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.7.0.3.0/package/templates/input.config-ranger.json.j2
>  6c5bb1f 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.7.0.3.0/package/templates/ranger_admin_pam.j2
>  d69ad6c 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.7.0.3.0/package/templates/ranger_remote_pam.j2
>  d69ad6c 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.7.0.3.0/package/templates/ranger_solr_jaas_conf.j2
>  a456688 
>   
> ambari-server/src/main/resou

Re: Review Request 60655: AMBARI-21405. Create custom action to force-remove packages

2017-07-05 Thread Alejandro Fernandez


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60655/#review179653
---


Fix it, then Ship it!





ambari-server/src/main/resources/custom_actions/scripts/force_remove_packages.py
Lines 27 (patched)


Add some documentation for how this is supposed to be used.


- Alejandro Fernandez


On July 5, 2017, 12:56 p.m., Attila Doroszlai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60655/
> ---
> 
> (Updated July 5, 2017, 12:56 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Sumit 
> Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-21405
> https://issues.apache.org/jira/browse/AMBARI-21405
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> * Add option in `Package` resource to remove package while leaving its 
> dependencies installed
> * Add custom action `force_remove_packages`
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/test/python/resource_management/TestPackageResource.py 
> 66227c66a7768aa044f30a1c76d782c5101be678 
>   
> ambari-common/src/main/python/resource_management/core/providers/package/__init__.py
>  21de1839c853794f252a4cb2265b89549a802d88 
>   
> ambari-common/src/main/python/resource_management/core/providers/package/apt.py
>  d095173f07c46842b6e9e38dc933e06a3e43a749 
>   
> ambari-common/src/main/python/resource_management/core/providers/package/choco.py
>  db552964279f509de90d2fde884bfc0fea12f7b0 
>   
> ambari-common/src/main/python/resource_management/core/providers/package/yumrpm.py
>  ea10a8601430b62bbe216848e1d26a5e8bc16927 
>   
> ambari-common/src/main/python/resource_management/core/providers/package/zypper.py
>  265c1626873bfb98196f102e73bb2dbedee94c80 
>   
> ambari-common/src/main/python/resource_management/core/resources/packaging.py 
> e3adc30f46abd3f33670f817421d88270b3e7ade 
>   
> ambari-server/src/main/resources/custom_action_definitions/system_action_definitions.xml
>  0f502568ddfb5da7d5574b2e33c6a354c37d9697 
>   
> ambari-server/src/main/resources/custom_actions/scripts/force_remove_packages.py
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/60655/diff/1/
> 
> 
> Testing
> ---
> 
> Tested custom action as part of cross-stack upgrade on CentOS7.
> 
> Manually tested custom action on Debian 7.6:
> 
> ```
> $ curl -X POST -d '{ "RequestInfo": { "action": "force_remove_packages", 
> "parameters": { "package_list": "vim-common" } } }' 
> http://localhost:8080/api/v1/clusters/TEST/requests
> 
> output.txt:
> 2017-07-05 12:29:59,400 - Package['vim-common'] {'action': ['remove'], 
> 'remove_with_dependencies': False}
> 2017-07-05 12:29:59,411 - Removing package vim-common ('/usr/bin/dpkg 
> --remove --ignore-depends vim-common vim-common')
> 2017-07-05 12:30:00,702 - Removed vim-common
> 
> $ apt-get install vim
> ...
> vim is already the newest version.
> You might want to run 'apt-get -f install' to correct these:
> The following packages have unmet dependencies:
>  vim : Depends: vim-common (= 2:7.3.547-7+deb7u3) but it is not going to be 
> installed
> ```
> 
> New unit tests for `Package` in `ambari-agent`:
> 
> ```
> Ran 456 tests in 38.256s
> 
> OK
> ```
> 
> 
> Thanks,
> 
> Attila Doroszlai
> 
>

Re: Review Request 60568: Log Feeder input config attribute "tail" should be clearer

2017-07-05 Thread Robert Nettleton


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60568/#review179640
---


Ship it!




Ship It!

- Robert Nettleton


On June 30, 2017, 3:13 p.m., Miklos Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60568/
> ---
> 
> (Updated June 30, 2017, 3:13 p.m.)
> 
> 
> Review request for Ambari, Oliver Szabo and Robert Nettleton.
> 
> 
> Bugs: AMBARI-21387
> https://issues.apache.org/jira/browse/AMBARI-21387
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> tail = true should mean that only the latest log file is loaded
> tail = false should mean that the previous rolled over log files are loaded 
> too, but not followed
> 
> AbstractInputFile must be prepared for previous rolled over files to check in 
> even when the next part is actively handled, as the checking in is done when 
> the data was moved to solr, so all file related fields were moved to maps, so 
> the checkin can identify which file's status is checked in.
> 
> Log files are sorted alphabetically after identified, thus ensureing that the 
> first one is the active one. In practice they were sorted already, but the 
> File.listFiles function doesn't ensures it according to it's documentation, 
> so it's better to do so explicitly.
> 
> The thread that is responsible for deleting unused checkpoint files should do 
> one more check before deleting a checkpoint file because the file it belongs 
> to is no more there, or it's file key doesn't matches the one in the 
> checkpoint file: first it should look up if there is any other file in the 
> same folder which have the same file key, in which case it is assumed that 
> the file was renamed since. As the code that's looking for a checkpoint file 
> to determine the next line to load does that by the name of the checkpoint 
> file, which is generated from the file key it will successfuly identify it 
> irrelevant of the name of the file.
> 
> I know, it's a bit complicated :) But I couldn't put it into words less 
> complicated.
> 
> 
> Also fixed Log Search Config ZK bug not to handle not node data change 
> related events.
> 
> 
> Diffs
> -
> 
>   
> ambari-logsearch/ambari-logsearch-config-zookeeper/src/main/java/org/apache/ambari/logsearch/config/zookeeper/LogSearchConfigZK.java
>  26375e1 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/AbstractInputFile.java
>  ab50eb7 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/Input.java
>  c36f96b 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/InputFile.java
>  fc40ca4 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/InputManager.java
>  19894ae 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/InputS3File.java
>  2b19503 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/test/java/org/apache/ambari/logfeeder/input/InputManagerTest.java
>  e9bbe7e 
> 
> 
> Diff: https://reviews.apache.org/r/60568/diff/1/
> 
> 
> Testing
> ---
> 
> Tested on local vagrant cluster, by loading previous files again and again, 
> and by renaming rolled over files. In both cases no duplicate data were 
> loaded, checkpoints were handled correctly.
> 
> 
> Thanks,
> 
> Miklos Gergely
> 
>

Re: Review Request 60637: Cleanup relevant Kerberos identities when a service is removed

2017-07-05 Thread Robert Levas



> On July 5, 2017, 11:21 a.m., Robert Levas wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/events/ServiceRemovedEvent.java
> > Lines 59 (patched)
> > 
> >
> > This is Java8-specific code.  Was it officially declared that Java7 is 
> > to be dropped for Ambari?
> 
> Attila Magyar wrote:
> I suppose yes because the ambari server is now compiled to 1.8 source 
> level on the trunk.

I see. The ambari-server pom file was updated yesterday to force JDK version 
1.8.

```
1.8
```

Dropping this issue.


- Robert


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60637/#review179634
---


On July 4, 2017, 11:20 a.m., Attila Magyar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60637/
> ---
> 
> (Updated July 4, 2017, 11:20 a.m.)
> 
> 
> Review request for Ambari, Balázs Bence Sári, Laszlo Puskas, Robert Levas, 
> and Sebastian Toader.
> 
> 
> Bugs: AMBARI-21392
> https://issues.apache.org/jira/browse/AMBARI-21392
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Upon removing a service from the cluster, the relevant Kerberos identities 
> should be removed as well. This includes any principals and keytab files.
> Care must be taken not to remove any principals or keytab files that are 
> still in use in the cluster.
> 
> entry point is: KerberosIdentityCleaner>>serviceRemoved
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/DeleteIdentityHandler.java
>  aa098b6 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelper.java
>  cc0c048 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelperImpl.java
>  b30f8f6 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/utilities/KerberosIdentityCleaner.java
>  0a8462f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/utilities/RemovableIdentities.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/utilities/UsedIdentities.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/events/ServiceComponentUninstalledEvent.java
>  5b55339 
>   
> ambari-server/src/main/java/org/apache/ambari/server/events/ServiceRemovedEvent.java
>  aca00a8 
>   ambari-server/src/main/java/org/apache/ambari/server/state/ServiceImpl.java 
> 5084703 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptorContainer.java
>  0a89c1d 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosComponentDescriptor.java
>  41d1f65 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosIdentityDescriptor.java
>  2023793 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/utilities/KerberosIdentityCleanerTest.java
>  d22c92e 
> 
> 
> Diff: https://reviews.apache.org/r/60637/diff/3/
> 
> 
> Testing
> ---
> 
> added new unittests.
> end2end tested manually:
>  - created a cluster with spark1 and spark2
>  - enabled kerberos
>  - removed spark1
>  - checked that spark identity was NOT removed (because it was still used by 
> spark2)
>  - removed spark2
>  - checked that the spark identity was removed
> 
> existing tests: PENDING
> 
> 
> Thanks,
> 
> Attila Magyar
> 
>

Re: Review Request 60637: Cleanup relevant Kerberos identities when a service is removed

2017-07-05 Thread Attila Magyar



> On July 5, 2017, 3:21 p.m., Robert Levas wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/events/ServiceRemovedEvent.java
> > Lines 59 (patched)
> > 
> >
> > This is Java8-specific code.  Was it officially declared that Java7 is 
> > to be dropped for Ambari?

I suppose yes because the ambari server is now compiled to 1.8 source level on 
the trunk.


- Attila


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60637/#review179634
---


On July 4, 2017, 3:20 p.m., Attila Magyar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60637/
> ---
> 
> (Updated July 4, 2017, 3:20 p.m.)
> 
> 
> Review request for Ambari, Balázs Bence Sári, Laszlo Puskas, Robert Levas, 
> and Sebastian Toader.
> 
> 
> Bugs: AMBARI-21392
> https://issues.apache.org/jira/browse/AMBARI-21392
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Upon removing a service from the cluster, the relevant Kerberos identities 
> should be removed as well. This includes any principals and keytab files.
> Care must be taken not to remove any principals or keytab files that are 
> still in use in the cluster.
> 
> entry point is: KerberosIdentityCleaner>>serviceRemoved
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/DeleteIdentityHandler.java
>  aa098b6 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelper.java
>  cc0c048 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelperImpl.java
>  b30f8f6 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/utilities/KerberosIdentityCleaner.java
>  0a8462f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/utilities/RemovableIdentities.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/utilities/UsedIdentities.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/events/ServiceComponentUninstalledEvent.java
>  5b55339 
>   
> ambari-server/src/main/java/org/apache/ambari/server/events/ServiceRemovedEvent.java
>  aca00a8 
>   ambari-server/src/main/java/org/apache/ambari/server/state/ServiceImpl.java 
> 5084703 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptorContainer.java
>  0a89c1d 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosComponentDescriptor.java
>  41d1f65 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosIdentityDescriptor.java
>  2023793 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/utilities/KerberosIdentityCleanerTest.java
>  d22c92e 
> 
> 
> Diff: https://reviews.apache.org/r/60637/diff/3/
> 
> 
> Testing
> ---
> 
> added new unittests.
> end2end tested manually:
>  - created a cluster with spark1 and spark2
>  - enabled kerberos
>  - removed spark1
>  - checked that spark identity was NOT removed (because it was still used by 
> spark2)
>  - removed spark2
>  - checked that the spark identity was removed
> 
> existing tests: PENDING
> 
> 
> Thanks,
> 
> Attila Magyar
> 
>

Re: Review Request 60637: Cleanup relevant Kerberos identities when a service is removed

2017-07-05 Thread Robert Levas


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60637/#review179634
---




ambari-server/src/main/java/org/apache/ambari/server/events/ServiceRemovedEvent.java
Lines 59 (patched)


This is Java8-specific code.  Was it officially declared that Java7 is to 
be dropped for Ambari?


- Robert Levas


On July 4, 2017, 11:20 a.m., Attila Magyar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60637/
> ---
> 
> (Updated July 4, 2017, 11:20 a.m.)
> 
> 
> Review request for Ambari, Balázs Bence Sári, Laszlo Puskas, Robert Levas, 
> and Sebastian Toader.
> 
> 
> Bugs: AMBARI-21392
> https://issues.apache.org/jira/browse/AMBARI-21392
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Upon removing a service from the cluster, the relevant Kerberos identities 
> should be removed as well. This includes any principals and keytab files.
> Care must be taken not to remove any principals or keytab files that are 
> still in use in the cluster.
> 
> entry point is: KerberosIdentityCleaner>>serviceRemoved
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/DeleteIdentityHandler.java
>  aa098b6 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelper.java
>  cc0c048 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelperImpl.java
>  b30f8f6 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/utilities/KerberosIdentityCleaner.java
>  0a8462f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/utilities/RemovableIdentities.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/utilities/UsedIdentities.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/events/ServiceComponentUninstalledEvent.java
>  5b55339 
>   
> ambari-server/src/main/java/org/apache/ambari/server/events/ServiceRemovedEvent.java
>  aca00a8 
>   ambari-server/src/main/java/org/apache/ambari/server/state/ServiceImpl.java 
> 5084703 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptorContainer.java
>  0a89c1d 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosComponentDescriptor.java
>  41d1f65 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosIdentityDescriptor.java
>  2023793 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/utilities/KerberosIdentityCleanerTest.java
>  d22c92e 
> 
> 
> Diff: https://reviews.apache.org/r/60637/diff/3/
> 
> 
> Testing
> ---
> 
> added new unittests.
> end2end tested manually:
>  - created a cluster with spark1 and spark2
>  - enabled kerberos
>  - removed spark1
>  - checked that spark identity was NOT removed (because it was still used by 
> spark2)
>  - removed spark2
>  - checked that the spark identity was removed
> 
> existing tests: PENDING
> 
> 
> Thanks,
> 
> Attila Magyar
> 
>

Re: Review Request 60634: Create property descriptions for internal Log Feeder configs

2017-07-05 Thread Robert Nettleton


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60634/#review179633
---


Ship it!




Ship It!

- Robert Nettleton


On July 4, 2017, 3:44 p.m., Miklos Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60634/
> ---
> 
> (Updated July 4, 2017, 3:44 p.m.)
> 
> 
> Review request for Ambari, Oliver Szabo and Robert Nettleton.
> 
> 
> Bugs: AMBARI-21387
> https://issues.apache.org/jira/browse/AMBARI-21387
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add property description to Log Feeder properties, similar to Log Search 
> propreties.
> 
> ---
> 
> Also removed some unnecessary properties:
> 
> foreground - the property didn't make much sense, the inputs are never 
> finished, they are always parsing continously the new log entries
> 
> input_config_dir / config.file - the config files (output + global) should be 
> specified at logfeeder.config.files, it is mandatory, there is no need for 3 
> different ways
> 
> user.home - we don't need a property to determine the user.home, and there is 
> no need for two fall back plans for the checkpoint folder
> 
> java.io.tmpdir - we already have another tmp dir property 
> (logfeeder.tmp.dir), there is no need for two
> 
> node.hostname - doesn't make sense to specify a different host name for 
> Metrics than for the Log Feeders
> 
> ---
> 
> Also fixed the few occasions where camel case name for the program wasn't 
> properly used (logfeeder and Logfeeder, instead of logFeeder and LogFeeder)
> 
> 
> Diffs
> -
> 
>   
> ambari-logsearch/ambari-logsearch-config-api/src/main/java/org/apache/ambari/logsearch/config/api/LogSearchConfig.java
>  ad1f5d4 
>   
> ambari-logsearch/ambari-logsearch-config-api/src/main/java/org/apache/ambari/logsearch/config/api/LogSearchConfigFactory.java
>  947e7e7 
>   
> ambari-logsearch/ambari-logsearch-config-api/src/test/java/org/apache/ambari/logsearch/config/api/LogSearchConfigClass1.java
>  7309382 
>   
> ambari-logsearch/ambari-logsearch-config-api/src/test/java/org/apache/ambari/logsearch/config/api/LogSearchConfigClass2.java
>  f83eeef 
>   
> ambari-logsearch/ambari-logsearch-config-api/src/test/java/org/apache/ambari/logsearch/config/api/LogSearchConfigFactoryTest.java
>  425694f 
>   
> ambari-logsearch/ambari-logsearch-config-zookeeper/src/main/java/org/apache/ambari/logsearch/config/zookeeper/LogSearchConfigZK.java
>  1926efa 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/LogFeeder.java
>  e7b6edc 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/common/ConfigHandler.java
>  25669d9 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/common/LogFeederException.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/common/LogfeederException.java
>  8a07602 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/filter/Filter.java
>  fd02497 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/filter/FilterGrok.java
>  50247e2 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/filter/FilterJSON.java
>  cfccdeb 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/filter/FilterKeyValue.java
>  f2a4186 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/AbstractInputFile.java
>  ab50eb7 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/Input.java
>  c36f96b 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/InputConfigUploader.java
>  8aec690 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/InputManager.java
>  19894ae 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/InputSimulate.java
>  5e7bdb3 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/loglevelfilter/LogLevelFilterHandler.java
>  8a4d953 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/metrics/LogFeederAMSClient.java
>  39526a5 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/metrics/MetricsManager.java
>  942c0b4 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/OutputHDFSFile.java
>  8f4b0b1 
>   
> ambari-logsearch/ambari-logsearch-logfeed

Re: Review Request 60543: DB consistency checker throws errors for missing 'parquet-logging' and 'product-info' configs after Ambari upgrade

2017-07-05 Thread Dmytro Grinenko


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60543/#review179631
---


Ship it!




Ship It!

- Dmytro Grinenko


On June 29, 2017, 4:28 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60543/
> ---
> 
> (Updated June 29, 2017, 4:28 p.m.)
> 
> 
> Review request for Ambari, Dmytro Grinenko and Jonathan Hurley.
> 
> 
> Bugs: AMBARI-21364
> https://issues.apache.org/jira/browse/AMBARI-21364
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> *STR*
> # Deployed cluster with Ambari version: 2.5.1.0-159 and HDP version: 
> 2.6.1.0-129
> # Upgrade Ambari to 2.5.2.0-74 (hash: 
> fd30644590991deb41241454d6e9091ed7a38e92)
> # Run "ambari-server start"
> 
> {code}
> root@ctr-e133-1493418528701-156570-01-05:/hwqe/hadoopqe# ambari-server 
> restart
> Using python  /usr/bin/python
> Restarting ambari-server
> Waiting for server stop...
> Ambari Server stopped
> Ambari Server running with administrator privileges.
> Organizing resource files at /var/lib/ambari-server/resources...
> Ambari database consistency check started...
> Server PID at: /var/run/ambari-server/ambari-server.pid
> Server out at: /var/log/ambari-server/ambari-server.out
> Server log at: /var/log/ambari-server/ambari-server.log
> Waiting for server start..
> DB configs consistency check failed. Run "ambari-server start 
> --skip-database-check" to skip. You may try --auto-fix-database flag to 
> attempt to fix issues automatically. If you use this "--skip-database-check" 
> option, do not make any changes to your cluster topology or perform a cluster 
> upgrade until you correct the database consistency issues. See 
> /var/log/ambari-server/ambari-server-check-database.log for more details on 
> the consistency issues.
> ERROR: Exiting with exit code -1.
> REASON: Ambari Server java process has stopped. Please check the logs for 
> more information.
> {code}
> 
> DB log: ambari-server-check-database.log
> {code}
> 2017-06-27 13:51:38,743  INFO - Executing query 'GET_SERVICES_WITH_CONFIGS'
> 2017-06-27 13:51:38,748  INFO - Comparing service configs from stack with 
> configs that we got from db
> 2017-06-27 13:51:38,748  INFO - Getting services from metainfo
> 2017-06-27 13:51:38,748  INFO - Processing HDP-2.6 / KAFKA
> 2017-06-27 13:51:38,748  INFO - Processing HDP-2.6 / PIG
> 2017-06-27 13:51:38,748  INFO - Processing HDP-2.6 / ZEPPELIN
> 2017-06-27 13:51:38,748  INFO - Processing HDP-2.6 / LOGSEARCH
> 2017-06-27 13:51:38,748  INFO - Processing HDP-2.6 / MAPREDUCE2
> 2017-06-27 13:51:38,748  INFO - Processing HDP-2.6 / SLIDER
> 2017-06-27 13:51:38,748  INFO - Processing HDP-2.6 / HIVE
> 2017-06-27 13:51:38,748  INFO - Processing HDP-2.6 / TEZ
> 2017-06-27 13:51:38,748  INFO - Processing HDP-2.6 / HBASE
> 2017-06-27 13:51:38,748  INFO - Processing HDP-2.6 / OOZIE
> 2017-06-27 13:51:38,748  INFO - Processing HDP-2.6 / FLUME
> 2017-06-27 13:51:38,748  INFO - Processing HDP-2.6 / MAHOUT
> 2017-06-27 13:51:38,748  INFO - Processing HDP-2.6 / HDFS
> 2017-06-27 13:51:38,748  INFO - Processing HDP-2.6 / DRUID
> 2017-06-27 13:51:38,748  INFO - Processing HDP-2.6 / AMBARI_METRICS
> 2017-06-27 13:51:38,749  INFO - Processing HDP-2.6 / SPARK
> 2017-06-27 13:51:38,749  INFO - Processing HDP-2.6 / SMARTSENSE
> 2017-06-27 13:51:38,749  INFO - Processing HDP-2.6 / AMBARI_INFRA
> 2017-06-27 13:51:38,749  INFO - Processing HDP-2.6 / YARN
> 2017-06-27 13:51:38,749  INFO - Processing HDP-2.6 / FALCON
> 2017-06-27 13:51:38,749  INFO - Processing HDP-2.6 / SPARK2
> 2017-06-27 13:51:38,749  INFO - Processing HDP-2.6 / ZOOKEEPER
> 2017-06-27 13:51:38,749  INFO - Processing HDP-2.6 / ATLAS
> 2017-06-27 13:51:38,749  INFO - Processing HDP-2.6 / SQOOP
> 2017-06-27 13:51:38,749  INFO - Processing HDP-2.6 / STORM
> 2017-06-27 13:51:38,749  INFO - Processing HDP-2.6 / KNOX
> 2017-06-27 13:51:38,749  INFO - Processing HDP-2.6 / KERBEROS
> 2017-06-27 13:51:38,749  INFO - Comparing required service configs from stack 
> with mapped service configs from db
> 2017-06-27 13:51:38,751 ERROR - Required config(s): product-info is(are) not 
> available for service SMARTSENSE with service config version 2 in cluster cl1
> 2017-06-27 13:51:38,751 ERROR - Required config(s): parquet-logging is(are) 
> not available for service HIVE with service config version 7 in cluster cl1
> 2017-06-27 13:51:38,751  INFO - Getting services which has mapped configs 
> which are not selected in clusterconfigmapping
> 2017-06-27 13:51:38,755  INFO - Checking Topology tables
> 2017-06-27 13:51:38,757  INFO - Checking for tables with large physical size
> 2017-06-27 13:51:38,759  INFO - The database table host_role_command is

Review Request 60655: AMBARI-21405. Create custom action to force-remove packages

2017-07-05 Thread Attila Doroszlai


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60655/
---

Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Sumit Mohanty, 
and Sid Wagle.


Bugs: AMBARI-21405
https://issues.apache.org/jira/browse/AMBARI-21405


Repository: ambari


Description
---

* Add option in `Package` resource to remove package while leaving its 
dependencies installed
* Add custom action `force_remove_packages`


Diffs
-

  ambari-agent/src/test/python/resource_management/TestPackageResource.py 
66227c66a7768aa044f30a1c76d782c5101be678 
  
ambari-common/src/main/python/resource_management/core/providers/package/__init__.py
 21de1839c853794f252a4cb2265b89549a802d88 
  
ambari-common/src/main/python/resource_management/core/providers/package/apt.py 
d095173f07c46842b6e9e38dc933e06a3e43a749 
  
ambari-common/src/main/python/resource_management/core/providers/package/choco.py
 db552964279f509de90d2fde884bfc0fea12f7b0 
  
ambari-common/src/main/python/resource_management/core/providers/package/yumrpm.py
 ea10a8601430b62bbe216848e1d26a5e8bc16927 
  
ambari-common/src/main/python/resource_management/core/providers/package/zypper.py
 265c1626873bfb98196f102e73bb2dbedee94c80 
  ambari-common/src/main/python/resource_management/core/resources/packaging.py 
e3adc30f46abd3f33670f817421d88270b3e7ade 
  
ambari-server/src/main/resources/custom_action_definitions/system_action_definitions.xml
 0f502568ddfb5da7d5574b2e33c6a354c37d9697 
  
ambari-server/src/main/resources/custom_actions/scripts/force_remove_packages.py
 PRE-CREATION 


Diff: https://reviews.apache.org/r/60655/diff/1/


Testing
---

Tested custom action as part of cross-stack upgrade on CentOS7.

Manually tested custom action on Debian 7.6:

```
$ curl -X POST -d '{ "RequestInfo": { "action": "force_remove_packages", 
"parameters": { "package_list": "vim-common" } } }' 
http://localhost:8080/api/v1/clusters/TEST/requests

output.txt:
2017-07-05 12:29:59,400 - Package['vim-common'] {'action': ['remove'], 
'remove_with_dependencies': False}
2017-07-05 12:29:59,411 - Removing package vim-common ('/usr/bin/dpkg --remove 
--ignore-depends vim-common vim-common')
2017-07-05 12:30:00,702 - Removed vim-common

$ apt-get install vim
...
vim is already the newest version.
You might want to run 'apt-get -f install' to correct these:
The following packages have unmet dependencies:
 vim : Depends: vim-common (= 2:7.3.547-7+deb7u3) but it is not going to be 
installed
```

New unit tests for `Package` in `ambari-agent`:

```
Ran 456 tests in 38.256s

OK
```


Thanks,

Attila Doroszlai

Re: Review Request 60543: DB consistency checker throws errors for missing 'parquet-logging' and 'product-info' configs after Ambari upgrade

2017-07-05 Thread Dmytro Grinenko



> On June 29, 2017, 5:13 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/metainfo.xml
> > Lines 232 (patched)
> > 
> >
> > Same change is needed in trunk for Hive in 3.0
> 
> Jonathan Hurley wrote:
> Ah - right - good catch... I keep forgetting we have essentially 2 
> versions of every service in trunk now.

nach, HIVE 3.0 linking to 2.1.0.3.0 which even not defining this configuration 
type


- Dmytro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60543/#review179277
---


On June 29, 2017, 4:28 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60543/
> ---
> 
> (Updated June 29, 2017, 4:28 p.m.)
> 
> 
> Review request for Ambari, Dmytro Grinenko and Jonathan Hurley.
> 
> 
> Bugs: AMBARI-21364
> https://issues.apache.org/jira/browse/AMBARI-21364
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> *STR*
> # Deployed cluster with Ambari version: 2.5.1.0-159 and HDP version: 
> 2.6.1.0-129
> # Upgrade Ambari to 2.5.2.0-74 (hash: 
> fd30644590991deb41241454d6e9091ed7a38e92)
> # Run "ambari-server start"
> 
> {code}
> root@ctr-e133-1493418528701-156570-01-05:/hwqe/hadoopqe# ambari-server 
> restart
> Using python  /usr/bin/python
> Restarting ambari-server
> Waiting for server stop...
> Ambari Server stopped
> Ambari Server running with administrator privileges.
> Organizing resource files at /var/lib/ambari-server/resources...
> Ambari database consistency check started...
> Server PID at: /var/run/ambari-server/ambari-server.pid
> Server out at: /var/log/ambari-server/ambari-server.out
> Server log at: /var/log/ambari-server/ambari-server.log
> Waiting for server start..
> DB configs consistency check failed. Run "ambari-server start 
> --skip-database-check" to skip. You may try --auto-fix-database flag to 
> attempt to fix issues automatically. If you use this "--skip-database-check" 
> option, do not make any changes to your cluster topology or perform a cluster 
> upgrade until you correct the database consistency issues. See 
> /var/log/ambari-server/ambari-server-check-database.log for more details on 
> the consistency issues.
> ERROR: Exiting with exit code -1.
> REASON: Ambari Server java process has stopped. Please check the logs for 
> more information.
> {code}
> 
> DB log: ambari-server-check-database.log
> {code}
> 2017-06-27 13:51:38,743  INFO - Executing query 'GET_SERVICES_WITH_CONFIGS'
> 2017-06-27 13:51:38,748  INFO - Comparing service configs from stack with 
> configs that we got from db
> 2017-06-27 13:51:38,748  INFO - Getting services from metainfo
> 2017-06-27 13:51:38,748  INFO - Processing HDP-2.6 / KAFKA
> 2017-06-27 13:51:38,748  INFO - Processing HDP-2.6 / PIG
> 2017-06-27 13:51:38,748  INFO - Processing HDP-2.6 / ZEPPELIN
> 2017-06-27 13:51:38,748  INFO - Processing HDP-2.6 / LOGSEARCH
> 2017-06-27 13:51:38,748  INFO - Processing HDP-2.6 / MAPREDUCE2
> 2017-06-27 13:51:38,748  INFO - Processing HDP-2.6 / SLIDER
> 2017-06-27 13:51:38,748  INFO - Processing HDP-2.6 / HIVE
> 2017-06-27 13:51:38,748  INFO - Processing HDP-2.6 / TEZ
> 2017-06-27 13:51:38,748  INFO - Processing HDP-2.6 / HBASE
> 2017-06-27 13:51:38,748  INFO - Processing HDP-2.6 / OOZIE
> 2017-06-27 13:51:38,748  INFO - Processing HDP-2.6 / FLUME
> 2017-06-27 13:51:38,748  INFO - Processing HDP-2.6 / MAHOUT
> 2017-06-27 13:51:38,748  INFO - Processing HDP-2.6 / HDFS
> 2017-06-27 13:51:38,748  INFO - Processing HDP-2.6 / DRUID
> 2017-06-27 13:51:38,748  INFO - Processing HDP-2.6 / AMBARI_METRICS
> 2017-06-27 13:51:38,749  INFO - Processing HDP-2.6 / SPARK
> 2017-06-27 13:51:38,749  INFO - Processing HDP-2.6 / SMARTSENSE
> 2017-06-27 13:51:38,749  INFO - Processing HDP-2.6 / AMBARI_INFRA
> 2017-06-27 13:51:38,749  INFO - Processing HDP-2.6 / YARN
> 2017-06-27 13:51:38,749  INFO - Processing HDP-2.6 / FALCON
> 2017-06-27 13:51:38,749  INFO - Processing HDP-2.6 / SPARK2
> 2017-06-27 13:51:38,749  INFO - Processing HDP-2.6 / ZOOKEEPER
> 2017-06-27 13:51:38,749  INFO - Processing HDP-2.6 / ATLAS
> 2017-06-27 13:51:38,749  INFO - Processing HDP-2.6 / SQOOP
> 2017-06-27 13:51:38,749  INFO - Processing HDP-2.6 / STORM
> 2017-06-27 13:51:38,749  INFO - Processing HDP-2.6 / KNOX
> 2017-06-27 13:51:38,749  INFO - Processing HDP-2.6 / KERBEROS
> 2017-06-27 13:51:38,749  INFO - Comparing required service configs from stack 
> with mapped service configs from db
> 2017-06-27 13:51:38,751 ERROR - Required config(s): product-info is(are) not 
> available for service SMARTSENSE with service config version 2 in clu

Re: Review Request 60645: Fix Unit Test Failures Caused by AMBARI-21391

2017-07-05 Thread Jonathan Hurley



> On July 5, 2017, 2:26 a.m., Attila Doroszlai wrote:
> > Ship It!
> 
> Attila Doroszlai wrote:
> Note that `ServicePropertiesTest` is still failing, although it's not 
> caused by AMBARI-21391.

Yes, there's an open Jira for that item. The BigInsight stack has a LOT of 
properties which need to be updated to match the schema correctly.


- Jonathan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60645/#review179599
---


On July 4, 2017, 11:45 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60645/
> ---
> 
> (Updated July 4, 2017, 11:45 p.m.)
> 
> 
> Review request for Ambari, Attila Doroszlai, Alejandro Fernandez, and Sid 
> Wagle.
> 
> 
> Bugs: AMBARI-21402
> https://issues.apache.org/jira/browse/AMBARI-21402
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Fix the following unit test failures caused by AMBARI-21391:
> 
> {code}
> Failed tests:
>   ConfigUpgradeValidityTest.testValidateConfigUpgradePacks:225 File 
> /Users/jhurley/src/apache/ambari/ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/config-upgrade.xml
>  appears to be a config upgrade pack, but does not define 
> 'upgrade-config.xsd' as its schema
> Tests in error:
>   TestActionDBAccessorImpl.testPersistActions:295 » Runtime 
> org.apache.ambari.se...
>   UpgradeResourceProviderHDP22Test.testCreateIntraStackUpgrade:305 » Runtime 
> org...
>   
> UpgradeActionTest.testFinalizeDowngradeWhenDidNotFinalizePreviousUpgrade:531->makeTwoUpgradesWhereLastDidNotComplete:248
>  » Ambari
>   ServicePropertiesTest.validatePropertySchemaOfServiceXMLs:50 » Ambari File 
> /Us...
> {code}
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/FinalizeUpgradeAction.java
>  8a54679379 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/config-upgrade.xml
>  540c017905 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/config-upgrade.xml
>  540c017905 
>   
> ambari-server/src/test/java/org/apache/ambari/server/actionmanager/TestActionDBAccessorImpl.java
>  88fc22d570 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderHDP22Test.java
>  d13d5ca085 
>   
> ambari-server/src/test/java/org/apache/ambari/server/serveraction/upgrades/UpgradeActionTest.java
>  a3031670f9 
> 
> 
> Diff: https://reviews.apache.org/r/60645/diff/1/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Re: Review Request 60650: Log Search UI: production build issues

2017-07-05 Thread Oleg Nechiporenko


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60650/#review179620
---


Ship it!




Ship It!

- Oleg Nechiporenko


On July 5, 2017, 11:01 a.m., Andrii Babiichuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60650/
> ---
> 
> (Updated July 5, 2017, 11:01 a.m.)
> 
> 
> Review request for Ambari and Oleg Nechiporenko.
> 
> 
> Bugs: AMBARI-21403
> https://issues.apache.org/jira/browse/AMBARI-21403
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> - Incorrect request header is sent on login
> - Login form is displayed to user on visiting the application page even if 
> authorization is active
> - AJAX requests are sent twice
> - `FilterButtonComponent` fails to be compiled: `ERROR in No template 
> specified for component FilterButtonComponent`
> - JS error after login: `ERROR TypeError: Cannot read property 'timeZone' of 
> undefined`, no content under navbar is displayed
> - JS error on logs loading: `ERROR TypeError: Cannot read property 'map' of 
> undefined`, no logs loaded as result
> - URL params in logs request aren't encoded
> 
> 
> Diffs
> -
> 
>   ambari-logsearch/ambari-logsearch-web-new/src/app/app.module.ts 1e0ebb0 
>   
> ambari-logsearch/ambari-logsearch-web-new/src/app/components/accordion-panel/accordion-panel.component.ts
>  85031b6 
>   
> ambari-logsearch/ambari-logsearch-web-new/src/app/components/app.component.spec.ts
>  3442132 
>   
> ambari-logsearch/ambari-logsearch-web-new/src/app/components/app.component.ts 
> c5a17c2 
>   
> ambari-logsearch/ambari-logsearch-web-new/src/app/components/dropdown-list/dropdown-list.component.ts
>  808fcf8 
>   
> ambari-logsearch/ambari-logsearch-web-new/src/app/components/filter-button/filter-button.component.ts
>  3da53ca 
>   
> ambari-logsearch/ambari-logsearch-web-new/src/app/components/filter-dropdown/filter-dropdown.component.spec.ts
>  9f4522b 
>   
> ambari-logsearch/ambari-logsearch-web-new/src/app/components/filter-dropdown/filter-dropdown.component.ts
>  84210dc 
>   
> ambari-logsearch/ambari-logsearch-web-new/src/app/components/filter-text-field/filter-text-field.component.spec.ts
>  740593f 
>   
> ambari-logsearch/ambari-logsearch-web-new/src/app/components/filter-text-field/filter-text-field.component.ts
>  857b511 
>   
> ambari-logsearch/ambari-logsearch-web-new/src/app/components/filters-panel/filters-panel.component.html
>  256b547 
>   
> ambari-logsearch/ambari-logsearch-web-new/src/app/components/filters-panel/filters-panel.component.ts
>  b2ae9b0 
>   
> ambari-logsearch/ambari-logsearch-web-new/src/app/components/login-form/login-form.component.html
>  37a96e9 
>   
> ambari-logsearch/ambari-logsearch-web-new/src/app/components/login-form/login-form.component.spec.ts
>  0498ee4 
>   
> ambari-logsearch/ambari-logsearch-web-new/src/app/components/login-form/login-form.component.ts
>  2a93775 
>   
> ambari-logsearch/ambari-logsearch-web-new/src/app/components/logs-list/logs-list.component.html
>  df72502 
>   
> ambari-logsearch/ambari-logsearch-web-new/src/app/components/logs-list/logs-list.component.ts
>  efa05e4 
>   
> ambari-logsearch/ambari-logsearch-web-new/src/app/components/main-container/main-container.component.html
>  61d1b93 
>   
> ambari-logsearch/ambari-logsearch-web-new/src/app/components/main-container/main-container.component.spec.ts
>  536f1ef 
>   
> ambari-logsearch/ambari-logsearch-web-new/src/app/components/main-container/main-container.component.ts
>  bac39ec 
>   
> ambari-logsearch/ambari-logsearch-web-new/src/app/components/menu-button/menu-button.component.spec.ts
>  d53677d 
>   
> ambari-logsearch/ambari-logsearch-web-new/src/app/components/menu-button/menu-button.component.ts
>  08b555c 
>   ambari-logsearch/ambari-logsearch-web-new/src/app/models/app-state.model.ts 
> PRE-CREATION 
>   ambari-logsearch/ambari-logsearch-web-new/src/app/models/store.model.ts 
> c169c1b 
>   
> ambari-logsearch/ambari-logsearch-web-new/src/app/services/component-actions.service.spec.ts
>  303fdab 
>   
> ambari-logsearch/ambari-logsearch-web-new/src/app/services/component-actions.service.ts
>  4cde0a0 
>   
> ambari-logsearch/ambari-logsearch-web-new/src/app/services/filtering.service.ts
>  9f6b7dc 
>   
> ambari-logsearch/ambari-logsearch-web-new/src/app/services/http-client.service.spec.ts
>  083ea23 
>   
> ambari-logsearch/ambari-logsearch-web-new/src/app/services/http-client.service.ts
>  cef6696 
>   
> ambari-logsearch/ambari-logsearch-web-new/src/app/services/mock-api-data.service.ts
>  757f68d 
>   
> ambari-logsearch/ambari-logsearch-web-new/src/app/services/storage/app-settings.service.ts
>  1c87a3c 
>   
> ambari-logsearch/ambari-logsearch-web-new/src/

Review Request 60650: Log Search UI: production build issues

2017-07-05 Thread Andrii Babiichuk


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60650/
---

Review request for Ambari and Oleg Nechiporenko.


Bugs: AMBARI-21403
https://issues.apache.org/jira/browse/AMBARI-21403


Repository: ambari


Description
---

- Incorrect request header is sent on login
- Login form is displayed to user on visiting the application page even if 
authorization is active
- AJAX requests are sent twice
- `FilterButtonComponent` fails to be compiled: `ERROR in No template specified 
for component FilterButtonComponent`
- JS error after login: `ERROR TypeError: Cannot read property 'timeZone' of 
undefined`, no content under navbar is displayed
- JS error on logs loading: `ERROR TypeError: Cannot read property 'map' of 
undefined`, no logs loaded as result
- URL params in logs request aren't encoded


Diffs
-

  ambari-logsearch/ambari-logsearch-web-new/src/app/app.module.ts 1e0ebb0 
  
ambari-logsearch/ambari-logsearch-web-new/src/app/components/accordion-panel/accordion-panel.component.ts
 85031b6 
  
ambari-logsearch/ambari-logsearch-web-new/src/app/components/app.component.spec.ts
 3442132 
  ambari-logsearch/ambari-logsearch-web-new/src/app/components/app.component.ts 
c5a17c2 
  
ambari-logsearch/ambari-logsearch-web-new/src/app/components/dropdown-list/dropdown-list.component.ts
 808fcf8 
  
ambari-logsearch/ambari-logsearch-web-new/src/app/components/filter-button/filter-button.component.ts
 3da53ca 
  
ambari-logsearch/ambari-logsearch-web-new/src/app/components/filter-dropdown/filter-dropdown.component.spec.ts
 9f4522b 
  
ambari-logsearch/ambari-logsearch-web-new/src/app/components/filter-dropdown/filter-dropdown.component.ts
 84210dc 
  
ambari-logsearch/ambari-logsearch-web-new/src/app/components/filter-text-field/filter-text-field.component.spec.ts
 740593f 
  
ambari-logsearch/ambari-logsearch-web-new/src/app/components/filter-text-field/filter-text-field.component.ts
 857b511 
  
ambari-logsearch/ambari-logsearch-web-new/src/app/components/filters-panel/filters-panel.component.html
 256b547 
  
ambari-logsearch/ambari-logsearch-web-new/src/app/components/filters-panel/filters-panel.component.ts
 b2ae9b0 
  
ambari-logsearch/ambari-logsearch-web-new/src/app/components/login-form/login-form.component.html
 37a96e9 
  
ambari-logsearch/ambari-logsearch-web-new/src/app/components/login-form/login-form.component.spec.ts
 0498ee4 
  
ambari-logsearch/ambari-logsearch-web-new/src/app/components/login-form/login-form.component.ts
 2a93775 
  
ambari-logsearch/ambari-logsearch-web-new/src/app/components/logs-list/logs-list.component.html
 df72502 
  
ambari-logsearch/ambari-logsearch-web-new/src/app/components/logs-list/logs-list.component.ts
 efa05e4 
  
ambari-logsearch/ambari-logsearch-web-new/src/app/components/main-container/main-container.component.html
 61d1b93 
  
ambari-logsearch/ambari-logsearch-web-new/src/app/components/main-container/main-container.component.spec.ts
 536f1ef 
  
ambari-logsearch/ambari-logsearch-web-new/src/app/components/main-container/main-container.component.ts
 bac39ec 
  
ambari-logsearch/ambari-logsearch-web-new/src/app/components/menu-button/menu-button.component.spec.ts
 d53677d 
  
ambari-logsearch/ambari-logsearch-web-new/src/app/components/menu-button/menu-button.component.ts
 08b555c 
  ambari-logsearch/ambari-logsearch-web-new/src/app/models/app-state.model.ts 
PRE-CREATION 
  ambari-logsearch/ambari-logsearch-web-new/src/app/models/store.model.ts 
c169c1b 
  
ambari-logsearch/ambari-logsearch-web-new/src/app/services/component-actions.service.spec.ts
 303fdab 
  
ambari-logsearch/ambari-logsearch-web-new/src/app/services/component-actions.service.ts
 4cde0a0 
  
ambari-logsearch/ambari-logsearch-web-new/src/app/services/filtering.service.ts 
9f6b7dc 
  
ambari-logsearch/ambari-logsearch-web-new/src/app/services/http-client.service.spec.ts
 083ea23 
  
ambari-logsearch/ambari-logsearch-web-new/src/app/services/http-client.service.ts
 cef6696 
  
ambari-logsearch/ambari-logsearch-web-new/src/app/services/mock-api-data.service.ts
 757f68d 
  
ambari-logsearch/ambari-logsearch-web-new/src/app/services/storage/app-settings.service.ts
 1c87a3c 
  
ambari-logsearch/ambari-logsearch-web-new/src/app/services/storage/app-state.service.ts
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-web-new/src/app/services/storage/audit-logs.service.ts
 7c322ed 
  
ambari-logsearch/ambari-logsearch-web-new/src/app/services/storage/bar-graphs.service.ts
 0109409 
  
ambari-logsearch/ambari-logsearch-web-new/src/app/services/storage/filters.service.ts
 b8748e9 
  
ambari-logsearch/ambari-logsearch-web-new/src/app/services/storage/graphs.service.ts
 eda04ee 
  
ambari-logsearch/ambari-logsearch-web-new/src/app/services/storage/nodes.service.ts
 7b2e6e9 
  
ambari-logsearch/ambari-logsearch-web-new/src/app/services/storage/s

Re: Review Request 60637: Cleanup relevant Kerberos identities when a service is removed

2017-07-05 Thread Attila Doroszlai


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60637/#review179614
---




ambari-server/src/main/java/org/apache/ambari/server/controller/utilities/UsedIdentities.java
Lines 1 (patched)


missing license


- Attila Doroszlai


On July 4, 2017, 5:20 p.m., Attila Magyar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60637/
> ---
> 
> (Updated July 4, 2017, 5:20 p.m.)
> 
> 
> Review request for Ambari, Balázs Bence Sári, Laszlo Puskas, Robert Levas, 
> and Sebastian Toader.
> 
> 
> Bugs: AMBARI-21392
> https://issues.apache.org/jira/browse/AMBARI-21392
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Upon removing a service from the cluster, the relevant Kerberos identities 
> should be removed as well. This includes any principals and keytab files.
> Care must be taken not to remove any principals or keytab files that are 
> still in use in the cluster.
> 
> entry point is: KerberosIdentityCleaner>>serviceRemoved
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/DeleteIdentityHandler.java
>  aa098b6 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelper.java
>  cc0c048 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelperImpl.java
>  b30f8f6 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/utilities/KerberosIdentityCleaner.java
>  0a8462f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/utilities/RemovableIdentities.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/utilities/UsedIdentities.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/events/ServiceComponentUninstalledEvent.java
>  5b55339 
>   
> ambari-server/src/main/java/org/apache/ambari/server/events/ServiceRemovedEvent.java
>  aca00a8 
>   ambari-server/src/main/java/org/apache/ambari/server/state/ServiceImpl.java 
> 5084703 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptorContainer.java
>  0a89c1d 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosComponentDescriptor.java
>  41d1f65 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosIdentityDescriptor.java
>  2023793 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/utilities/KerberosIdentityCleanerTest.java
>  d22c92e 
> 
> 
> Diff: https://reviews.apache.org/r/60637/diff/3/
> 
> 
> Testing
> ---
> 
> added new unittests.
> end2end tested manually:
>  - created a cluster with spark1 and spark2
>  - enabled kerberos
>  - removed spark1
>  - checked that spark identity was NOT removed (because it was still used by 
> spark2)
>  - removed spark2
>  - checked that the spark identity was removed
> 
> existing tests: PENDING
> 
> 
> Thanks,
> 
> Attila Magyar
> 
>

40 matches

Mail list logo