Re: Review Request 50995: Collect JVM Heap, GC and thread pool metrics from Ambari Server and push to AMS

2016-08-19 Thread Alejandro Fernandez 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50995/#review145539
---




ambari-server/src/main/java/org/apache/ambari/server/metrics/system/impl/AmbariMetricSinkImpl.java
 (line 31)


Add javadoc for what this class does.



ambari-server/src/main/java/org/apache/ambari/server/metrics/system/impl/MetricsServiceImpl.java
 (line 47)


Add javadoc


- Alejandro Fernandez


On Aug. 18, 2016, 6:03 p.m., Li-Wei Tseng wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50995/
> ---
> 
> (Updated Aug. 18, 2016, 6:03 p.m.)
> 
> 
> Review request for Ambari and Aravindan Vijayan.
> 
> 
> Bugs: AMBARI-17591
> https://issues.apache.org/jira/browse/AMBARI-17591
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Created metrics service which has a JVM metrics source and a sink. The JVM 
> metrics source collect JVM metrics and pass it to the sink which then publish 
> the metrics to AMS
> 
> 
> Diffs
> -
> 
>   ambari-server/conf/unix/metrics.properties PRE-CREATION 
>   ambari-server/pom.xml 0d91458 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  097f01c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java
>  256f0d1 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metrics/system/AmbariMetricSink.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metrics/system/MetricsService.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metrics/system/MetricsSource.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metrics/system/impl/AbstractMetricsSource.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metrics/system/impl/AmbariMetricSinkImpl.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metrics/system/impl/Configuration.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metrics/system/impl/JvmMetricsSource.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metrics/system/impl/MetricsServiceImpl.java
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/files/grafana-dashboards/default/grafana-ambari-server.json
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/50995/diff/
> 
> 
> Testing
> ---
> 
> Manually tested it.
> 
> 
> Thanks,
> 
> Li-Wei Tseng
> 
>

Re: Review Request 51245: Both NameNodes reporting version mismatches before Finalizing RU

2016-08-19 Thread Alejandro Fernandez 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51245/#review146231
---




ambari-server/src/main/java/org/apache/ambari/server/stack/MasterHostResolver.java
 (line 144)


Can't we change the getNameNodePair() function instead to use a different 
method if the cluster is Kerberized?
This way, we prevent future bugs from calls to getNameNodePair


- Alejandro Fernandez


On Aug. 19, 2016, 5:58 p.m., Nate Cole wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51245/
> ---
> 
> (Updated Aug. 19, 2016, 5:58 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez and Jonathan Hurley.
> 
> 
> Bugs: AMBARI-18216
> https://issues.apache.org/jira/browse/AMBARI-18216
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When doing RU on a secured cluster, the MasterHostResolver class is failing 
> the lookup of JMX. Ambari's SSL configuration must be used just like metrics 
> do.
> 
> Also adding some logging, and making sure that if we have 2 confirmed NN, 
> that we are always picking one to be the active and one to be the standby.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/MasterHostResolver.java
>  e6154d2 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java 
> 6cefa79 
>   ambari-server/src/main/java/org/apache/ambari/server/utils/HTTPUtils.java 
> cfb7128 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/UpgradeHelperTest.java
>  55cb23b 
> 
> Diff: https://reviews.apache.org/r/51245/diff/
> 
> 
> Testing
> ---
> 
> Manual.  Automated pending.
> 
> 
> Thanks,
> 
> Nate Cole
> 
>

Re: Review Request 51254: Kafka listeners property does not show SASL_PLAINTEXT protocol when Kerberos is enabled

2016-08-19 Thread Anita Jebaraj 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51254/
---

(Updated Aug. 19, 2016, 9:55 p.m.)


Review request for Ambari, Di Li, Jonathan Hurley, Nate Cole, Robert Levas, 
Sumit Mohanty, Sriharsha Chintalapani, and Vitalyi Brodetskyi.


Bugs: AMBARI-17694
https://issues.apache.org/jira/browse/AMBARI-17694


Repository: ambari


Description (updated)
---

This is a continuation of the review request 
https://reviews.apache.org/r/50047/ . Opening a new request for clarity.

When kerberos is enabled, the protocol for listeners in 
/etc/kafka/conf/server.properties is updated from PLAINTEXT to PLAINTEXTSASL, 
even though the Ambari UI shows otherwise


Updated the patch based on comments from Sumit, A follow up JIRA Ambari-17929 
was reverted earlier, I have included the code for upgrade from the patch with 
some minor changes. Also included the code for validation in stack_advisor 
based on which a warning will be thrown in the UI, when the listeners and 
security.inter.broker.protocol values are not in sync.

"Stack advisor code to recommend changes to revert to PLAINTEXT if not 
kerberized" --> All the values get reverted back to default when kerberos is 
disabled, so I didn't make any changes to this.


Including Sumit's comment here for reference.

sorry, had to revert it. After deployment and some user operations the 
configurations went out of sync

...
listeners=PLAINTEXT://nat-r7-kxqs-xaagents-re-3.openstacklocal:6667,SSL://nat-r7-kxqs-xaagents-re-3.openstacklocal:
security.inter.broker.protocol=PLAINTEXTSASL
...

The over all approach is sound - works for fresh deployments blueprint and UI. 
Looked through the patch and here are some additional changes (by the way, I am 
not very familiar with Kafka):

Existing deployments (that will go through Ambari upgrade to 2.4.0) will 
either need 
1) code to replace PLAINTEXT to PLAINTEXTSASL in kafka.py or, 
2) UpgradeCatalog code to fix the configs stored in the DB. The later is a 
better approach.

Stack advisor code to ensure "listeners" and 
"security.inter.broker.protocol" values are in sync. E.g. error if one is 
PLAINTEXTSASL and one isn't
Stack advisor code to recommend changes to revert to PLAINTEXT if not 
kerberized. I did not try but I was not sure if config will revert back 
properly when unkerberized.

Sorry, could not get to it during code review.

Can we move this JIRA to 2.5.0, next release. It appears that some more test 
scenarios need to be covered. Its too close for the 2.4.0 release to get all 
paths tested.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/VariableReplacementHelper.java
 66be3bf 
  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
 12553a5 
  ambari-server/src/main/resources/common-services/KAFKA/0.10.0/kerberos.json 
e1e6461 
  
ambari-server/src/main/resources/common-services/KAFKA/0.8.1/package/scripts/kafka.py
 9066ab5 
  ambari-server/src/main/resources/common-services/KAFKA/0.9.0/kerberos.json 
2b1c01b 
  ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 
64e8e03 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/VariableReplacementHelperTest.java
 ee2a671 
  
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog240Test.java
 854ce7d 

Diff: https://reviews.apache.org/r/51254/diff/


Testing
---

Added 2 new test case,
 Ran mvn test
 Tested in Ambari UI, by enabling kerberos, listeners protocol is updated and 
kafka started successfully
 
 Deployed Ambari 2.2.2 enabled kerberos and then did an update, the kafka 
listeners value got updated
 
 Changed the value in the UI, a warning is thrown on save.

 Disabled kerberos, value got reset to default PLAINTEXT://localhost:6667
 
 
 
 Sumit, please let me know if any other scenarios need to be tested.


File Attachments


validation.jpg
  
https://reviews.apache.org/media/uploaded/files/2016/08/19/286c8651-2954-41ba-8d53-b734129ff1a8__validation.jpg


Thanks,

Anita Jebaraj

Re: Review Request 51254: Kafka listeners property does not show SASL_PLAINTEXT protocol when Kerberos is enabled

2016-08-19 Thread Anita Jebaraj 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51254/
---

(Updated Aug. 19, 2016, 9:49 p.m.)


Review request for Ambari, Di Li, Jonathan Hurley, Nate Cole, Robert Levas, 
Sumit Mohanty, and Sriharsha Chintalapani.


Bugs: AMBARI-17694
https://issues.apache.org/jira/browse/AMBARI-17694


Repository: ambari


Description
---

This is a continuation of the review request 
https://reviews.apache.org/r/50047/ . Opening a new request for clarity.

When kerberos is enabled, the protocol for listeners in 
/etc/kafka/conf/server.properties is updated from PLAINTEXT to PLAINTEXTSASL, 
even though the Ambari UI shows otherwise


Updated the patch based on comments from Sumit, A follow up JIRA Ambari-17929 
was reverted earlier, included the code for upgrade from the patch with some 
minor changes. Included code for validation in stack_advisor based on which a 
warning will be thrown in the UI, when the listeners and 
security.inter.broker.protocol values are not in sync.

"Stack advisor code to recommend changes to revert to PLAINTEXT if not 
kerberized" --> All the values get reverted back to default when kerberos is 
disabled, so I didn't make any changes to this.


Including Sumit's comment here for reference.

sorry, had to revert it. After deployment and some user operations the 
configurations went out of sync

...
listeners=PLAINTEXT://nat-r7-kxqs-xaagents-re-3.openstacklocal:6667,SSL://nat-r7-kxqs-xaagents-re-3.openstacklocal:
security.inter.broker.protocol=PLAINTEXTSASL
...

The over all approach is sound - works for fresh deployments blueprint and UI. 
Looked through the patch and here are some additional changes (by the way, I am 
not very familiar with Kafka):

Existing deployments (that will go through Ambari upgrade to 2.4.0) will 
either need 
1) code to replace PLAINTEXT to PLAINTEXTSASL in kafka.py or, 
2) UpgradeCatalog code to fix the configs stored in the DB. The later is a 
better approach.

Stack advisor code to ensure "listeners" and 
"security.inter.broker.protocol" values are in sync. E.g. error if one is 
PLAINTEXTSASL and one isn't
Stack advisor code to recommend changes to revert to PLAINTEXT if not 
kerberized. I did not try but I was not sure if config will revert back 
properly when unkerberized.

Sorry, could not get to it during code review.

Can we move this JIRA to 2.5.0, next release. It appears that some more test 
scenarios need to be covered. Its too close for the 2.4.0 release to get all 
paths tested.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/VariableReplacementHelper.java
 66be3bf 
  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
 12553a5 
  ambari-server/src/main/resources/common-services/KAFKA/0.10.0/kerberos.json 
e1e6461 
  
ambari-server/src/main/resources/common-services/KAFKA/0.8.1/package/scripts/kafka.py
 9066ab5 
  ambari-server/src/main/resources/common-services/KAFKA/0.9.0/kerberos.json 
2b1c01b 
  ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 
64e8e03 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/VariableReplacementHelperTest.java
 ee2a671 
  
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog240Test.java
 854ce7d 

Diff: https://reviews.apache.org/r/51254/diff/


Testing
---

Added 2 new test case,
 Ran mvn test
 Tested in Ambari UI, by enabling kerberos, listeners protocol is updated and 
kafka started successfully
 
 Deployed Ambari 2.2.2 enabled kerberos and then did an update, the kafka 
listeners value got updated
 
 Changed the value in the UI, a warning is thrown on save.

 Disabled kerberos, value got reset to default PLAINTEXT://localhost:6667
 
 
 
 Sumit, please let me know if any other scenarios need to be tested.


File Attachments (updated)


validation.jpg
  
https://reviews.apache.org/media/uploaded/files/2016/08/19/286c8651-2954-41ba-8d53-b734129ff1a8__validation.jpg


Thanks,

Anita Jebaraj

Review Request 51254: Kafka listeners property does not show SASL_PLAINTEXT protocol when Kerberos is enabled

2016-08-19 Thread Anita Jebaraj 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51254/
---

Review request for Ambari, Di Li, Jonathan Hurley, Nate Cole, Robert Levas, 
Sumit Mohanty, and Sriharsha Chintalapani.


Bugs: AMBARI-17694
https://issues.apache.org/jira/browse/AMBARI-17694


Repository: ambari


Description
---

This is a continuation of the review request 
https://reviews.apache.org/r/50047/ . Opening a new request for clarity.

When kerberos is enabled, the protocol for listeners in 
/etc/kafka/conf/server.properties is updated from PLAINTEXT to PLAINTEXTSASL, 
even though the Ambari UI shows otherwise


Updated the patch based on comments from Sumit, A follow up JIRA Ambari-17929 
was reverted earlier, included the code for upgrade from the patch with some 
minor changes. Included code for validation in stack_advisor based on which a 
warning will be thrown in the UI, when the listeners and 
security.inter.broker.protocol values are not in sync.

"Stack advisor code to recommend changes to revert to PLAINTEXT if not 
kerberized" --> All the values get reverted back to default when kerberos is 
disabled, so I didn't make any changes to this.


Including Sumit's comment here for reference.

sorry, had to revert it. After deployment and some user operations the 
configurations went out of sync

...
listeners=PLAINTEXT://nat-r7-kxqs-xaagents-re-3.openstacklocal:6667,SSL://nat-r7-kxqs-xaagents-re-3.openstacklocal:
security.inter.broker.protocol=PLAINTEXTSASL
...

The over all approach is sound - works for fresh deployments blueprint and UI. 
Looked through the patch and here are some additional changes (by the way, I am 
not very familiar with Kafka):

Existing deployments (that will go through Ambari upgrade to 2.4.0) will 
either need 
1) code to replace PLAINTEXT to PLAINTEXTSASL in kafka.py or, 
2) UpgradeCatalog code to fix the configs stored in the DB. The later is a 
better approach.

Stack advisor code to ensure "listeners" and 
"security.inter.broker.protocol" values are in sync. E.g. error if one is 
PLAINTEXTSASL and one isn't
Stack advisor code to recommend changes to revert to PLAINTEXT if not 
kerberized. I did not try but I was not sure if config will revert back 
properly when unkerberized.

Sorry, could not get to it during code review.

Can we move this JIRA to 2.5.0, next release. It appears that some more test 
scenarios need to be covered. Its too close for the 2.4.0 release to get all 
paths tested.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/VariableReplacementHelper.java
 66be3bf 
  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
 12553a5 
  ambari-server/src/main/resources/common-services/KAFKA/0.10.0/kerberos.json 
e1e6461 
  
ambari-server/src/main/resources/common-services/KAFKA/0.8.1/package/scripts/kafka.py
 9066ab5 
  ambari-server/src/main/resources/common-services/KAFKA/0.9.0/kerberos.json 
2b1c01b 
  ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 
64e8e03 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/VariableReplacementHelperTest.java
 ee2a671 
  
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog240Test.java
 854ce7d 

Diff: https://reviews.apache.org/r/51254/diff/


Testing
---

Added 2 new test case,
 Ran mvn test
 Tested in Ambari UI, by enabling kerberos, listeners protocol is updated and 
kafka started successfully
 
 Deployed Ambari 2.2.2 enabled kerberos and then did an update, the kafka 
listeners value got updated
 
 Changed the value in the UI, a warning is thrown on save.

 Disabled kerberos, value got reset to default PLAINTEXT://localhost:6667
 
 
 
 Sumit, please let me know if any other scenarios need to be tested.


Thanks,

Anita Jebaraj

Re: Review Request 51143: Alert on Atlas after adding it to a secure cluster as HBase table initialization fails

2016-08-19 Thread Alejandro Fernandez 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51143/#review146218
---



Pushed to trunk, commit c17d98013cf3ae9649f85345c3dc16fae5d8e994
branch-2.4, commit e662dcb7c82ee3fa5ebad2b4d61d1459dd71a0f2

- Alejandro Fernandez


On Aug. 19, 2016, 8:27 p.m., Vitalyi Brodetskyi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51143/
> ---
> 
> (Updated Aug. 19, 2016, 8:27 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Andrew Onischuk, Dmitro 
> Lisnichenko, Dmytro Sen, Myroslav Papirkovskyy, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-18165
> https://issues.apache.org/jira/browse/AMBARI-18165
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> {code}
> Traceback (most recent call last):
>   File 
> "/var/lib/ambari-agent/cache/common-services/HBASE/0.96.0.2.0/package/scripts/hbase_regionserver.py",
>  line 198, in 
> HbaseRegionServer().execute()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 280, in execute
> method(env)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 720, in restart
> self.start(env, upgrade_type=upgrade_type)
>   File 
> "/var/lib/ambari-agent/cache/common-services/HBASE/0.96.0.2.0/package/scripts/hbase_regionserver.py",
>  line 124, in start
> self.post_start(env, upgrade_type=upgrade_type)
>   File 
> "/var/lib/ambari-agent/cache/common-services/HBASE/0.96.0.2.0/package/scripts/hbase_regionserver.py",
>  line 89, in post_start
> self.apply_atlas_acl(params.hbase_user)
>   File 
> "/var/lib/ambari-agent/cache/common-services/HBASE/0.96.0.2.0/package/scripts/hbase_regionserver.py",
>  line 114, in apply_atlas_acl
> shell.checked_call(format("{kinit_cmd}; {perm_cmd}"), 
> user=params.hbase_user, tries=10, try_sleep=10)
>   File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
> line 71, in inner
> result = function(command, **kwargs)
>   File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
> line 93, in checked_call
> tries=tries, try_sleep=try_sleep)
>   File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
> line 141, in _call_wrapper
> result = _call(command, **kwargs_copy)
>   File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
> line 294, in _call
> raise Fail(err_msg)
> resource_management.core.exceptions.Fail: Execution of '/usr/bin/kinit -kt 
> /etc/security/keytabs/hbase.service.keytab 
> hbase/nat-r6-gjss-ambari-blueprints-4re-1-1.openstacklo...@example.com; echo 
> "grant 'atlas', 'RWXCA', 'atlas_titan'" | hbase shell -n' returned 1. 
>  Hortonworks #
> This is MOTD message, added for testing in qe infra
> ERROR ArgumentError: Can't find a table: atlas_titan
> {code}
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/constants.py
>  9306a43 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata.py
>  7e9c0f6 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata_server.py
>  43b8fb2 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py
>  ab4af79 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/status_params.py
>  76575f6 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/templates/atlas_hbase_setup.rb.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/hbase_regionserver.py
>  a22fe94 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_features.json
>  a754cd5 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/metainfo.xml 
> 4ba59d5 
>   ambari-server/src/test/python/stacks/2.0.6/HBASE/test_hbase_regionserver.py 
> a6c8e0f 
>   ambari-server/src/test/python/stacks/2.3/ATLAS/test_metadata_server.py 
> 44841f4 
>   ambari-server/src/test/python/stacks/2.5/ATLAS/test_atlas_server.py 7fdf3e6 
> 
> Diff: https://reviews.apache.org/r/51143/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>

Re: Review Request 51143: Alert on Atlas after adding it to a secure cluster as HBase table initialization fails

2016-08-19 Thread Alejandro Fernandez 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51143/#review146217
---


Ship it!




Ship It!

- Alejandro Fernandez


On Aug. 19, 2016, 8:27 p.m., Vitalyi Brodetskyi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51143/
> ---
> 
> (Updated Aug. 19, 2016, 8:27 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Andrew Onischuk, Dmitro 
> Lisnichenko, Dmytro Sen, Myroslav Papirkovskyy, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-18165
> https://issues.apache.org/jira/browse/AMBARI-18165
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> {code}
> Traceback (most recent call last):
>   File 
> "/var/lib/ambari-agent/cache/common-services/HBASE/0.96.0.2.0/package/scripts/hbase_regionserver.py",
>  line 198, in 
> HbaseRegionServer().execute()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 280, in execute
> method(env)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 720, in restart
> self.start(env, upgrade_type=upgrade_type)
>   File 
> "/var/lib/ambari-agent/cache/common-services/HBASE/0.96.0.2.0/package/scripts/hbase_regionserver.py",
>  line 124, in start
> self.post_start(env, upgrade_type=upgrade_type)
>   File 
> "/var/lib/ambari-agent/cache/common-services/HBASE/0.96.0.2.0/package/scripts/hbase_regionserver.py",
>  line 89, in post_start
> self.apply_atlas_acl(params.hbase_user)
>   File 
> "/var/lib/ambari-agent/cache/common-services/HBASE/0.96.0.2.0/package/scripts/hbase_regionserver.py",
>  line 114, in apply_atlas_acl
> shell.checked_call(format("{kinit_cmd}; {perm_cmd}"), 
> user=params.hbase_user, tries=10, try_sleep=10)
>   File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
> line 71, in inner
> result = function(command, **kwargs)
>   File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
> line 93, in checked_call
> tries=tries, try_sleep=try_sleep)
>   File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
> line 141, in _call_wrapper
> result = _call(command, **kwargs_copy)
>   File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
> line 294, in _call
> raise Fail(err_msg)
> resource_management.core.exceptions.Fail: Execution of '/usr/bin/kinit -kt 
> /etc/security/keytabs/hbase.service.keytab 
> hbase/nat-r6-gjss-ambari-blueprints-4re-1-1.openstacklo...@example.com; echo 
> "grant 'atlas', 'RWXCA', 'atlas_titan'" | hbase shell -n' returned 1. 
>  Hortonworks #
> This is MOTD message, added for testing in qe infra
> ERROR ArgumentError: Can't find a table: atlas_titan
> {code}
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/constants.py
>  9306a43 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata.py
>  7e9c0f6 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata_server.py
>  43b8fb2 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py
>  ab4af79 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/status_params.py
>  76575f6 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/templates/atlas_hbase_setup.rb.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/hbase_regionserver.py
>  a22fe94 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_features.json
>  a754cd5 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/metainfo.xml 
> 4ba59d5 
>   ambari-server/src/test/python/stacks/2.0.6/HBASE/test_hbase_regionserver.py 
> a6c8e0f 
>   ambari-server/src/test/python/stacks/2.3/ATLAS/test_metadata_server.py 
> 44841f4 
>   ambari-server/src/test/python/stacks/2.5/ATLAS/test_atlas_server.py 7fdf3e6 
> 
> Diff: https://reviews.apache.org/r/51143/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>

Re: Review Request 51143: Alert on Atlas after adding it to a secure cluster as HBase table initialization fails

2016-08-19 Thread Vitalyi Brodetskyi 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51143/
---

(Updated Сер. 19, 2016, 8:27 після полудня)


Review request for Ambari, Alejandro Fernandez, Andrew Onischuk, Dmitro 
Lisnichenko, Dmytro Sen, Myroslav Papirkovskyy, and Sumit Mohanty.


Bugs: AMBARI-18165
https://issues.apache.org/jira/browse/AMBARI-18165


Repository: ambari


Description
---

{code}
Traceback (most recent call last):
  File 
"/var/lib/ambari-agent/cache/common-services/HBASE/0.96.0.2.0/package/scripts/hbase_regionserver.py",
 line 198, in 
HbaseRegionServer().execute()
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
 line 280, in execute
method(env)
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
 line 720, in restart
self.start(env, upgrade_type=upgrade_type)
  File 
"/var/lib/ambari-agent/cache/common-services/HBASE/0.96.0.2.0/package/scripts/hbase_regionserver.py",
 line 124, in start
self.post_start(env, upgrade_type=upgrade_type)
  File 
"/var/lib/ambari-agent/cache/common-services/HBASE/0.96.0.2.0/package/scripts/hbase_regionserver.py",
 line 89, in post_start
self.apply_atlas_acl(params.hbase_user)
  File 
"/var/lib/ambari-agent/cache/common-services/HBASE/0.96.0.2.0/package/scripts/hbase_regionserver.py",
 line 114, in apply_atlas_acl
shell.checked_call(format("{kinit_cmd}; {perm_cmd}"), 
user=params.hbase_user, tries=10, try_sleep=10)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
line 71, in inner
result = function(command, **kwargs)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
line 93, in checked_call
tries=tries, try_sleep=try_sleep)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
line 141, in _call_wrapper
result = _call(command, **kwargs_copy)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
line 294, in _call
raise Fail(err_msg)
resource_management.core.exceptions.Fail: Execution of '/usr/bin/kinit -kt 
/etc/security/keytabs/hbase.service.keytab 
hbase/nat-r6-gjss-ambari-blueprints-4re-1-1.openstacklo...@example.com; echo 
"grant 'atlas', 'RWXCA', 'atlas_titan'" | hbase shell -n' returned 1.  
Hortonworks #
This is MOTD message, added for testing in qe infra
ERROR ArgumentError: Can't find a table: atlas_titan
{code}


Diffs (updated)
-

  
ambari-common/src/main/python/resource_management/libraries/functions/constants.py
 9306a43 
  
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata.py
 7e9c0f6 
  
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata_server.py
 43b8fb2 
  
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py
 ab4af79 
  
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/status_params.py
 76575f6 
  
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/templates/atlas_hbase_setup.rb.j2
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/hbase_regionserver.py
 a22fe94 
  
ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_features.json
 a754cd5 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/metainfo.xml 
4ba59d5 
  ambari-server/src/test/python/stacks/2.0.6/HBASE/test_hbase_regionserver.py 
a6c8e0f 
  ambari-server/src/test/python/stacks/2.3/ATLAS/test_metadata_server.py 
44841f4 
  ambari-server/src/test/python/stacks/2.5/ATLAS/test_atlas_server.py 7fdf3e6 

Diff: https://reviews.apache.org/r/51143/diff/


Testing
---

mvn clean test


Thanks,

Vitalyi Brodetskyi

Re: Review Request 51241: PREVIEW - RU: Storm components were stopped during RU and can not be started

2016-08-19 Thread Nate Cole 


> On Aug. 19, 2016, 11:16 a.m., Nate Cole wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/StormUpgradeKerberosDescriptorConfig.java,
> >  line 81
> > 
> >
> > Probably use a common function for this (have to make it public:  
> > KerberosHelperImpl.getKerberosDescriptorUpdates(Cluster cluster)
> > 
> > Have to inject the KerberosHelper.
> 
> Dmitro Lisnichenko wrote:
> We would in any case need ArtifactDAO here (at least to write new value). 
> Extracted a separate method
> 
> Jonathan Hurley wrote:
> I agree with Nate here; you should use the helper method to get the exact 
> right cluster. Otherwise, you're updating all of them. I know we don't really 
> support multi-cluster, but we shouldn't just do all of them.

I'm not saying you can't use the ArtifactDAO to merge - but use the 
KerberosHelperImpl method to get the ONLY entity for the current cluster.  Your 
code iterates all of them, updating the property for all entities, whether it's 
for the cluster of interest or not.


- Nate


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51241/#review146182
---


On Aug. 19, 2016, 12:54 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51241/
> ---
> 
> (Updated Aug. 19, 2016, 12:54 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Jayush Luniya, and Nate Cole.
> 
> 
> Bugs: AMBARI-18213
> https://issues.apache.org/jira/browse/AMBARI-18213
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> STR:
> # Install cluster 2.4.2.0-258 on Ambari 2.2.2.0
> # Enable HA
> # Enable security
> # Upgrade ambari to 240
> # Perform RU to 2.5.0.0-1208
> 
> Deeper study shows that kerberos descriptor json in database ("artifact" 
> table) still contains values and properties that are actual for 2.4 stack.
> So the issue workflow should look like:
> - Old stack version is installed
> - Kerberos descriptor gets saved to database
> - Security is enabled
> - Stack upgrade is performed
> - Keytab regeneration is performed, and it populates service config with 
> obsolete property values
> 
> The issue happens on "Stack upgrade is performed" step. We never update 
> kerberos descriptor json in database to correspond to a new stack.
> 
> From nimbus.out
> {code}Exception in thread "main" java.lang.ExceptionInInitializerError
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:270)
> at clojure.lang.RT.classForName(RT.java:2154)
> at clojure.lang.RT.classForName(RT.java:2163)
> at clojure.lang.RT.loadClassForName(RT.java:2182)
> at clojure.lang.RT.load(RT.java:436)
> at clojure.lang.RT.load(RT.java:412)
> at clojure.core$load$fn__5448.invoke(core.clj:5866)
> at clojure.core$load.doInvoke(core.clj:5865)
> at clojure.lang.RestFn.invoke(RestFn.java:408)
> at clojure.core$load_one.invoke(core.clj:5671)
> at clojure.core$load_lib$fn__5397.invoke(core.clj:5711)
> at clojure.core$load_lib.doInvoke(core.clj:5710)
> at clojure.lang.RestFn.applyTo(RestFn.java:142)
> at clojure.core$apply.invoke(core.clj:632)
> at clojure.core$load_libs.doInvoke(core.clj:5749)
> at clojure.lang.RestFn.applyTo(RestFn.java:137)
> at clojure.core$apply.invoke(core.clj:632)
> at clojure.core$require.doInvoke(core.clj:5832)
> at clojure.lang.RestFn.invoke(RestFn.java:408)
> at 
> org.apache.storm.daemon.nimbus$loading__5340__auto8560.invoke(nimbus.clj:16)
> at org.apache.storm.daemon.nimbus__init.load(Unknown Source)
> at org.apache.storm.daemon.nimbus__init.(Unknown Source)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:270)
> at clojure.lang.RT.classForName(RT.java:2154)
> at clojure.lang.RT.classForName(RT.java:2163)
> at clojure.lang.RT.loadClassForName(RT.java:2182)
> at clojure.lang.RT.load(RT.java:436)
> at clojure.lang.RT.load(RT.java:412)
> at clojure.core$load$fn__5448.invoke(core.clj:5866)
> at clojure.core$load.doInvoke(core.clj:5865)
> at clojure.lang.RestFn.invoke(RestFn.java:408)
> at clojure.lang.Var.invoke(Var.java:379)
> at org.apache.storm.daemon.nimbus.(Unknown Source)
> Caused by: java.lang.ClassNotFoundException: 
> backtype.storm.security.auth.authorizer.SimpleACLAuthorizer
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at

Re: Review Request 51241: PREVIEW - RU: Storm components were stopped during RU and can not be started

2016-08-19 Thread Jonathan Hurley 


> On Aug. 19, 2016, 11:16 a.m., Nate Cole wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/StormUpgradeKerberosDescriptorConfig.java,
> >  line 81
> > 
> >
> > Probably use a common function for this (have to make it public:  
> > KerberosHelperImpl.getKerberosDescriptorUpdates(Cluster cluster)
> > 
> > Have to inject the KerberosHelper.
> 
> Dmitro Lisnichenko wrote:
> We would in any case need ArtifactDAO here (at least to write new value). 
> Extracted a separate method

I agree with Nate here; you should use the helper method to get the exact right 
cluster. Otherwise, you're updating all of them. I know we don't really support 
multi-cluster, but we shouldn't just do all of them.


- Jonathan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51241/#review146182
---


On Aug. 19, 2016, 12:54 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51241/
> ---
> 
> (Updated Aug. 19, 2016, 12:54 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Jayush Luniya, and Nate Cole.
> 
> 
> Bugs: AMBARI-18213
> https://issues.apache.org/jira/browse/AMBARI-18213
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> STR:
> # Install cluster 2.4.2.0-258 on Ambari 2.2.2.0
> # Enable HA
> # Enable security
> # Upgrade ambari to 240
> # Perform RU to 2.5.0.0-1208
> 
> Deeper study shows that kerberos descriptor json in database ("artifact" 
> table) still contains values and properties that are actual for 2.4 stack.
> So the issue workflow should look like:
> - Old stack version is installed
> - Kerberos descriptor gets saved to database
> - Security is enabled
> - Stack upgrade is performed
> - Keytab regeneration is performed, and it populates service config with 
> obsolete property values
> 
> The issue happens on "Stack upgrade is performed" step. We never update 
> kerberos descriptor json in database to correspond to a new stack.
> 
> From nimbus.out
> {code}Exception in thread "main" java.lang.ExceptionInInitializerError
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:270)
> at clojure.lang.RT.classForName(RT.java:2154)
> at clojure.lang.RT.classForName(RT.java:2163)
> at clojure.lang.RT.loadClassForName(RT.java:2182)
> at clojure.lang.RT.load(RT.java:436)
> at clojure.lang.RT.load(RT.java:412)
> at clojure.core$load$fn__5448.invoke(core.clj:5866)
> at clojure.core$load.doInvoke(core.clj:5865)
> at clojure.lang.RestFn.invoke(RestFn.java:408)
> at clojure.core$load_one.invoke(core.clj:5671)
> at clojure.core$load_lib$fn__5397.invoke(core.clj:5711)
> at clojure.core$load_lib.doInvoke(core.clj:5710)
> at clojure.lang.RestFn.applyTo(RestFn.java:142)
> at clojure.core$apply.invoke(core.clj:632)
> at clojure.core$load_libs.doInvoke(core.clj:5749)
> at clojure.lang.RestFn.applyTo(RestFn.java:137)
> at clojure.core$apply.invoke(core.clj:632)
> at clojure.core$require.doInvoke(core.clj:5832)
> at clojure.lang.RestFn.invoke(RestFn.java:408)
> at 
> org.apache.storm.daemon.nimbus$loading__5340__auto8560.invoke(nimbus.clj:16)
> at org.apache.storm.daemon.nimbus__init.load(Unknown Source)
> at org.apache.storm.daemon.nimbus__init.(Unknown Source)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:270)
> at clojure.lang.RT.classForName(RT.java:2154)
> at clojure.lang.RT.classForName(RT.java:2163)
> at clojure.lang.RT.loadClassForName(RT.java:2182)
> at clojure.lang.RT.load(RT.java:436)
> at clojure.lang.RT.load(RT.java:412)
> at clojure.core$load$fn__5448.invoke(core.clj:5866)
> at clojure.core$load.doInvoke(core.clj:5865)
> at clojure.lang.RestFn.invoke(RestFn.java:408)
> at clojure.lang.Var.invoke(Var.java:379)
> at org.apache.storm.daemon.nimbus.(Unknown Source)
> Caused by: java.lang.ClassNotFoundException: 
> backtype.storm.security.auth.authorizer.SimpleACLAuthorizer
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:190)
> at 
> org.apache.storm.daemon.common$mk_authorization_handler.invoke(common.clj:412)
> at org.apache.storm.ui.core__init.load(Unknown Source)
> at

Re: Review Request 51242: Restify logsearch endpoints

2016-08-19 Thread Oliver Szabo 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51242/
---

(Updated Aug. 19, 2016, 7:57 p.m.)


Review request for Ambari, Dharmesh Makwana, Miklos Gergely, Robert Nettleton, 
and Sumit Mohanty.


Bugs: AMBARI-18214
https://issues.apache.org/jira/browse/AMBARI-18214


Repository: ambari


Description
---

- restify endpoints as possible
- rename dashboard to service
- there are still some endpoints what is not too resty, but that is the "vol 1" 
for refactoring. in some cases, some of the endpoints can be merged into one, 
and call a same method with just different params. for now, i kept them, i did 
not want to do any changes in the business code part. (after Miklos is ready 
with the refactoring, we can do some changes there too, and get rid some weird 
names in the API ... like verbs e.g.: /service/logs/aggregated -> it can be 
also in /service/logs endpoint and add aggregated as a query param etc.)


Diffs
-

  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/SolrDaoBase.java
 672507a 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/doc/DocConstants.java
 c1572b7 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/manager/AuditMgr.java
 d4f2986 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/manager/LogsMgr.java
 b03a643 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/rest/AuditLogsREST.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/rest/AuditREST.java
 5ed49fd 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/rest/DashboardREST.java
 0144edc 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/rest/LogFileREST.java
 d53cff9 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/rest/PublicREST.java
 af48acd 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/rest/ServiceLogsREST.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/rest/UserConfigREST.java
 4b1675f 
  ambari-logsearch/ambari-logsearch-portal/src/main/webapp/login.html 44f1aeb 
  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/collection_bases/VAuditLogListBase.js
 12f7c31 
  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/collection_bases/VEventHistoryListBase.js
 f6f720d 
  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/collection_bases/VGroupListBase.js
 a34aaa3 
  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/collection_bases/VLogLevelListBase.js
 59b5ae8 
  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/collection_bases/VLogListBase.js
 7b102d5 
  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/collection_bases/VNameValueListBase.js
 d59eaa2 
  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/collection_bases/VNodeListBase.js
 7c7dcf8 
  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/model_bases/VAuditLogBase.js
 1283875 
  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/model_bases/VCommonModelBase.js
 4723e3e 
  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/model_bases/VEventHistoryBase.js
 c237ade 
  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/model_bases/VGraphInfoBase.js
 a707629 
  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/model_bases/VLogLevelBase.js
 0384bc2 
  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/model_bases/VUserFilterBase.js
 35171aa 
  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/utils/ViewUtils.js
 331ffd6 
  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/views/audit/AuditAggregatedView.js
 c04aaf9 
  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/views/audit/AuditTabLayoutView.js
 0b570ac 
  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/views/dashboard/BubbleGraphTableLayoutView.js
 42b94d5 
  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/views/dashboard/ComponentListView.js
 abd3740 
  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/views/dashboard/ComponentsView.js
 66cc277 
  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/views/dashboard/DashboardView.js
 c3fd9c2 
  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/views/dashboard/GridTableLayoutView.js
 1cbdef8 
  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/views/dashboard/HostsView.js
 dd82130

Re: Review Request 51207: Wrong hostname in timeline.metrics.service.webapp.address breaks AMS HA

2016-08-19 Thread Sid Wagle 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51207/#review146203
---



The path does everythong expected for the 0.0.0.0:6188 revert and the alert 
definition, however not sure about the param.py changes for 1 host these should 
not be required since we should be gettign all collectors as 
host1:port1,host:2port2 with the changes done for HA.

- Sid Wagle


On Aug. 18, 2016, 3:46 p.m., Dmytro Sen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51207/
> ---
> 
> (Updated Aug. 18, 2016, 3:46 p.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan and Sid Wagle.
> 
> 
> Bugs: AMBARI-18199
> https://issues.apache.org/jira/browse/AMBARI-18199
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> On adding additional Collector the webapp address contains wrong hostname.
> 
> java.net.BindException: Port in use: ambari-sid-5.com:6188
> at 
> org.apache.hadoop.http.HttpServer2.openListeners(HttpServer2.java:919)
> at org.apache.hadoop.http.HttpServer2.start(HttpServer2.java:856)
> at 
> org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:374)
> at 
> org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer.startWebApp(ApplicationHistoryServer.java:180)
> at 
> org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer.serviceStart(ApplicationHistoryServer.java:92)
> at 
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at 
> org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer.launchAppHistoryServer(ApplicationHistoryServer.java:138)
> at 
> org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer.main(ApplicationHistoryServer.java:147)
> Caused by: java.net.BindException: Cannot assign requested address
> 
> Stop storing collector hostnames in timeline.metrics.service.webapp.address 
> property
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/test/python/ambari_agent/TestAlerts.py e114daa 
>   ambari-common/src/main/python/ambari_commons/ambari_metrics_helper.py 
> PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/ACCUMULO/1.6.1.2.2.0/package/scripts/params.py
>  7352f33 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml
>  c02df71 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py
>  fc95aa7 
>   
> ambari-server/src/main/resources/common-services/FLUME/1.4.0.2.0/package/scripts/params.py
>  fe00140 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py
>  f3643b1 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/alerts/alert_metrics_deviation.py
>  f0a1d5c 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py
>  37e2426 
>   
> ambari-server/src/main/resources/common-services/KAFKA/0.8.1/package/scripts/params.py
>  d5451b2 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/params_linux.py
>  d0ac3a4 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/params.py
>  699c51f 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
> 7133c27 
>   
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/hooks/before-START/scripts/params.py
>  5bd2a9e 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
>  2c32f7c 
>   
> ambari-server/src/test/python/stacks/2.0.6/HDFS/test_alert_metrics_deviation.py
>  5bfbfc6 
>   ambari-server/src/test/python/stacks/2.0.6/configs/default.json 28624fd 
>   
> ambari-server/src/test/python/stacks/2.0.6/configs/default_ams_embedded.json 
> ceedb67 
>   
> ambari-server/src/test/python/stacks/2.0.6/configs/default_hive_non_hdfs.json 
> 3692691 
>   ambari-server/src/test/python/stacks/2.0.6/configs/default_with_bucket.json 
> 54172f3 
>   ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py 
> 081a66a 
>   
> ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade-hdfs-secure.json
>  7b07d87 
>   ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade.json 
> cf84bb7 
>   ambari-server/src/test/python/stacks/2.2/configs/ranger-admin-upgrade.json 
> 6874db5 
>   
> ambari-server/src/test/python/stacks/2.2/configs/ranger-usersync-upgrade.json 
> 0f5d647 
>   ambari-server/src/test/python/stacks/2.3/configs/ats_1_5.json c64ca14 
>

Re: Review Request 51241: PREVIEW - RU: Storm components were stopped during RU and can not be started

2016-08-19 Thread Jayush Luniya 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51241/#review146202
---


Ship it!




Ship It!

- Jayush Luniya


On Aug. 19, 2016, 4:54 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51241/
> ---
> 
> (Updated Aug. 19, 2016, 4:54 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Jayush Luniya, and Nate Cole.
> 
> 
> Bugs: AMBARI-18213
> https://issues.apache.org/jira/browse/AMBARI-18213
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> STR:
> # Install cluster 2.4.2.0-258 on Ambari 2.2.2.0
> # Enable HA
> # Enable security
> # Upgrade ambari to 240
> # Perform RU to 2.5.0.0-1208
> 
> Deeper study shows that kerberos descriptor json in database ("artifact" 
> table) still contains values and properties that are actual for 2.4 stack.
> So the issue workflow should look like:
> - Old stack version is installed
> - Kerberos descriptor gets saved to database
> - Security is enabled
> - Stack upgrade is performed
> - Keytab regeneration is performed, and it populates service config with 
> obsolete property values
> 
> The issue happens on "Stack upgrade is performed" step. We never update 
> kerberos descriptor json in database to correspond to a new stack.
> 
> From nimbus.out
> {code}Exception in thread "main" java.lang.ExceptionInInitializerError
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:270)
> at clojure.lang.RT.classForName(RT.java:2154)
> at clojure.lang.RT.classForName(RT.java:2163)
> at clojure.lang.RT.loadClassForName(RT.java:2182)
> at clojure.lang.RT.load(RT.java:436)
> at clojure.lang.RT.load(RT.java:412)
> at clojure.core$load$fn__5448.invoke(core.clj:5866)
> at clojure.core$load.doInvoke(core.clj:5865)
> at clojure.lang.RestFn.invoke(RestFn.java:408)
> at clojure.core$load_one.invoke(core.clj:5671)
> at clojure.core$load_lib$fn__5397.invoke(core.clj:5711)
> at clojure.core$load_lib.doInvoke(core.clj:5710)
> at clojure.lang.RestFn.applyTo(RestFn.java:142)
> at clojure.core$apply.invoke(core.clj:632)
> at clojure.core$load_libs.doInvoke(core.clj:5749)
> at clojure.lang.RestFn.applyTo(RestFn.java:137)
> at clojure.core$apply.invoke(core.clj:632)
> at clojure.core$require.doInvoke(core.clj:5832)
> at clojure.lang.RestFn.invoke(RestFn.java:408)
> at 
> org.apache.storm.daemon.nimbus$loading__5340__auto8560.invoke(nimbus.clj:16)
> at org.apache.storm.daemon.nimbus__init.load(Unknown Source)
> at org.apache.storm.daemon.nimbus__init.(Unknown Source)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:270)
> at clojure.lang.RT.classForName(RT.java:2154)
> at clojure.lang.RT.classForName(RT.java:2163)
> at clojure.lang.RT.loadClassForName(RT.java:2182)
> at clojure.lang.RT.load(RT.java:436)
> at clojure.lang.RT.load(RT.java:412)
> at clojure.core$load$fn__5448.invoke(core.clj:5866)
> at clojure.core$load.doInvoke(core.clj:5865)
> at clojure.lang.RestFn.invoke(RestFn.java:408)
> at clojure.lang.Var.invoke(Var.java:379)
> at org.apache.storm.daemon.nimbus.(Unknown Source)
> Caused by: java.lang.ClassNotFoundException: 
> backtype.storm.security.auth.authorizer.SimpleACLAuthorizer
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:190)
> at 
> org.apache.storm.daemon.common$mk_authorization_handler.invoke(common.clj:412)
> at org.apache.storm.ui.core__init.load(Unknown Source)
> at org.apache.storm.ui.core__init.(Unknown Source)
> ... 35 more
> {code}
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/StormUpgradeKerberosDescriptorConfig.java
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml
>  65a7880 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> bc8e9f9 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml
>  98e5038 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> e886301 
> 
> Diff: https://reviews.apache.org/r/51241/diff/
> 
> 
> Testing
> ---
> 
> As of now, testing on live cluster
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Re: Review Request 51245: Both NameNodes reporting version mismatches before Finalizing RU

2016-08-19 Thread Nate Cole 


> On Aug. 19, 2016, 1:10 p.m., Jonathan Hurley wrote:
> > The code in UpgradeHelper is still a bit fragile (even though now you're 
> > preventing the if-statement from failed. I say we either log the fact that 
> > we're not scheduling restarts or throw an exception:
> > 
> >   // Special case for NAMENODE when there are multiple
> >   if (service.serviceName.equalsIgnoreCase("HDFS") && 
> > component.equalsIgnoreCase("NAMENODE")) {
> > 
> > // Rolling Upgrade requires first upgrading the Standby, then 
> > the Active NameNode.
> > // Whereas NonRolling needs to do the following:
> > //   NameNode HA:  Pick one to the be active, and the other the 
> > standby.
> > //   Non-NameNode HA: Upgrade first the SECONDARY, then the 
> > primary NAMENODE
> > switch (upgradePack.getType()) {
> >   case ROLLING:
> > if (!hostsType.hosts.isEmpty() && hostsType.master != null 
> > && hostsType.secondary != null) { 
> >   // The order is important, first do the standby, then the 
> > active namenode.
> >   LinkedHashSet order = new LinkedHashSet();
> > 
> >   order.add(hostsType.secondary);
> >   order.add(hostsType.master);
> > 
> >   // Override the hosts with the ordered collection
> >   hostsType.hosts = order;
> > 
> >   builder.add(context, hostsType, service.serviceName,
> >   svc.isClientOnlyService(), pc, null);
> > }
> > break;

I think it's appropriate to throw an Exception.  Will add.


- Nate


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51245/#review146196
---


On Aug. 19, 2016, 12:55 p.m., Nate Cole wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51245/
> ---
> 
> (Updated Aug. 19, 2016, 12:55 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez and Jonathan Hurley.
> 
> 
> Bugs: AMBARI-18216
> https://issues.apache.org/jira/browse/AMBARI-18216
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When doing RU on a secured cluster, the MasterHostResolver class is failing 
> the lookup of JMX. Ambari's SSL configuration must be used just like metrics 
> do.
> 
> Also adding some logging, and making sure that if we have 2 confirmed NN, 
> that we are always picking one to be the active and one to be the standby.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/MasterHostResolver.java
>  e6154d2 
>   ambari-server/src/main/java/org/apache/ambari/server/utils/HTTPUtils.java 
> cfb7128 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/UpgradeHelperTest.java
>  55cb23b 
> 
> Diff: https://reviews.apache.org/r/51245/diff/
> 
> 
> Testing
> ---
> 
> Manual.  Automated pending.
> 
> 
> Thanks,
> 
> Nate Cole
> 
>

Re: Review Request 51242: Restify logsearch endpoints

2016-08-19 Thread Robert Nettleton 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51242/#review146193
---


Fix it, then Ship it!




Overall, the change looks like a good addition/change to the LogSearch REST API.

I have a few issues I've raised below, regarding the build-level dependencies 
and Apache licensing concerns.

Thanks.


ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/rest/ServiceLogsREST.java
 (line 29)


Maybe I've missed something, but I can't find the build-level changes to 
introduce Swagger into the Ambari build.  

Is this change introduced in a separate patch?



ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/rest/ServiceLogsREST.java
 (line 30)


Another thing to consider:

Are we sure that all of the Swagger components required by this change in 
Ambari are using the correct Apache licensing? 

Some initial searches I've done online seem to indicate that this code is 
Apache-licensed, but we'll probably need to verify that for all 
usages/inclusions of Swagger in Ambari.


- Robert Nettleton


On Aug. 19, 2016, 3:26 p.m., Oliver Szabo wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51242/
> ---
> 
> (Updated Aug. 19, 2016, 3:26 p.m.)
> 
> 
> Review request for Ambari, Dharmesh Makwana, Miklos Gergely, Robert 
> Nettleton, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-18214
> https://issues.apache.org/jira/browse/AMBARI-18214
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> - restify endpoints as possible
> - rename dashboard to service
> - there are still some endpoints what is not too resty, but that is the "vol 
> 1" for refactoring. in some cases, some of the endpoints can be merged into 
> one, and call a same method with just different params. for now, i kept them, 
> i did not want to do any changes in the business code part. (after Miklos is 
> ready with the refactoring, we can do some changes there too, and get rid 
> some weird names in the API ... like verbs e.g.: /service/logs/aggregated -> 
> it can be also in /service/logs endpoint and add aggregated as a query param 
> etc.)
> 
> 
> Diffs
> -
> 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/SolrDaoBase.java
>  672507a 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/doc/DocConstants.java
>  c1572b7 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/manager/AuditMgr.java
>  d4f2986 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/manager/LogsMgr.java
>  b03a643 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/rest/AuditLogsREST.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/rest/AuditREST.java
>  5ed49fd 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/rest/DashboardREST.java
>  0144edc 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/rest/LogFileREST.java
>  d53cff9 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/rest/PublicREST.java
>  af48acd 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/rest/ServiceLogsREST.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/rest/UserConfigREST.java
>  4b1675f 
>   ambari-logsearch/ambari-logsearch-portal/src/main/webapp/login.html 44f1aeb 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/collection_bases/VAuditLogListBase.js
>  12f7c31 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/collection_bases/VEventHistoryListBase.js
>  f6f720d 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/collection_bases/VGroupListBase.js
>  a34aaa3 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/collection_bases/VLogLevelListBase.js
>  59b5ae8 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/collection_bases/VLogListBase.js
>  7b102d5 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/collection_bases/VNameValueListBase.js
>  d59eaa2 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/collection_bases/VNodeListBase.js
>  7c7dcf8 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/model_bases/VAuditLogBase.js
>

Re: Review Request 51241: PREVIEW - RU: Storm components were stopped during RU and can not be started

2016-08-19 Thread Dmitro Lisnichenko 


> On Aug. 19, 2016, 6:16 p.m., Nate Cole wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/StormUpgradeKerberosDescriptorConfig.java,
> >  line 81
> > 
> >
> > Probably use a common function for this (have to make it public:  
> > KerberosHelperImpl.getKerberosDescriptorUpdates(Cluster cluster)
> > 
> > Have to inject the KerberosHelper.

We would in any case need ArtifactDAO here (at least to write new value). 
Extracted a separate method


- Dmitro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51241/#review146182
---


On Aug. 19, 2016, 7:54 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51241/
> ---
> 
> (Updated Aug. 19, 2016, 7:54 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Jayush Luniya, and Nate Cole.
> 
> 
> Bugs: AMBARI-18213
> https://issues.apache.org/jira/browse/AMBARI-18213
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> STR:
> # Install cluster 2.4.2.0-258 on Ambari 2.2.2.0
> # Enable HA
> # Enable security
> # Upgrade ambari to 240
> # Perform RU to 2.5.0.0-1208
> 
> Deeper study shows that kerberos descriptor json in database ("artifact" 
> table) still contains values and properties that are actual for 2.4 stack.
> So the issue workflow should look like:
> - Old stack version is installed
> - Kerberos descriptor gets saved to database
> - Security is enabled
> - Stack upgrade is performed
> - Keytab regeneration is performed, and it populates service config with 
> obsolete property values
> 
> The issue happens on "Stack upgrade is performed" step. We never update 
> kerberos descriptor json in database to correspond to a new stack.
> 
> From nimbus.out
> {code}Exception in thread "main" java.lang.ExceptionInInitializerError
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:270)
> at clojure.lang.RT.classForName(RT.java:2154)
> at clojure.lang.RT.classForName(RT.java:2163)
> at clojure.lang.RT.loadClassForName(RT.java:2182)
> at clojure.lang.RT.load(RT.java:436)
> at clojure.lang.RT.load(RT.java:412)
> at clojure.core$load$fn__5448.invoke(core.clj:5866)
> at clojure.core$load.doInvoke(core.clj:5865)
> at clojure.lang.RestFn.invoke(RestFn.java:408)
> at clojure.core$load_one.invoke(core.clj:5671)
> at clojure.core$load_lib$fn__5397.invoke(core.clj:5711)
> at clojure.core$load_lib.doInvoke(core.clj:5710)
> at clojure.lang.RestFn.applyTo(RestFn.java:142)
> at clojure.core$apply.invoke(core.clj:632)
> at clojure.core$load_libs.doInvoke(core.clj:5749)
> at clojure.lang.RestFn.applyTo(RestFn.java:137)
> at clojure.core$apply.invoke(core.clj:632)
> at clojure.core$require.doInvoke(core.clj:5832)
> at clojure.lang.RestFn.invoke(RestFn.java:408)
> at 
> org.apache.storm.daemon.nimbus$loading__5340__auto8560.invoke(nimbus.clj:16)
> at org.apache.storm.daemon.nimbus__init.load(Unknown Source)
> at org.apache.storm.daemon.nimbus__init.(Unknown Source)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:270)
> at clojure.lang.RT.classForName(RT.java:2154)
> at clojure.lang.RT.classForName(RT.java:2163)
> at clojure.lang.RT.loadClassForName(RT.java:2182)
> at clojure.lang.RT.load(RT.java:436)
> at clojure.lang.RT.load(RT.java:412)
> at clojure.core$load$fn__5448.invoke(core.clj:5866)
> at clojure.core$load.doInvoke(core.clj:5865)
> at clojure.lang.RestFn.invoke(RestFn.java:408)
> at clojure.lang.Var.invoke(Var.java:379)
> at org.apache.storm.daemon.nimbus.(Unknown Source)
> Caused by: java.lang.ClassNotFoundException: 
> backtype.storm.security.auth.authorizer.SimpleACLAuthorizer
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:190)
> at 
> org.apache.storm.daemon.common$mk_authorization_handler.invoke(common.clj:412)
> at org.apache.storm.ui.core__init.load(Unknown Source)
> at org.apache.storm.ui.core__init.(Unknown Source)
> ... 35 more
> {code}
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/StormUpgradeKerberosDescriptorConfig.java
>  PRE-CREATION 
>   
>

Review Request 51245: Both NameNodes reporting version mismatches before Finalizing RU

2016-08-19 Thread Nate Cole 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51245/
---

Review request for Ambari, Alejandro Fernandez and Jonathan Hurley.


Bugs: AMBARI-18216
https://issues.apache.org/jira/browse/AMBARI-18216


Repository: ambari


Description
---

When doing RU on a secured cluster, the MasterHostResolver class is failing the 
lookup of JMX. Ambari's SSL configuration must be used just like metrics do.

Also adding some logging, and making sure that if we have 2 confirmed NN, that 
we are always picking one to be the active and one to be the standby.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/stack/MasterHostResolver.java
 e6154d2 
  ambari-server/src/main/java/org/apache/ambari/server/utils/HTTPUtils.java 
cfb7128 
  
ambari-server/src/test/java/org/apache/ambari/server/state/UpgradeHelperTest.java
 55cb23b 

Diff: https://reviews.apache.org/r/51245/diff/


Testing
---

Manual.  Automated pending.


Thanks,

Nate Cole

Re: Review Request 51241: PREVIEW - RU: Storm components were stopped during RU and can not be started

2016-08-19 Thread Dmitro Lisnichenko 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51241/
---

(Updated Aug. 19, 2016, 7:54 p.m.)


Review request for Ambari, Jonathan Hurley, Jayush Luniya, and Nate Cole.


Changes
---

Refectored implementation according to comments


Bugs: AMBARI-18213
https://issues.apache.org/jira/browse/AMBARI-18213


Repository: ambari


Description
---

STR:
# Install cluster 2.4.2.0-258 on Ambari 2.2.2.0
# Enable HA
# Enable security
# Upgrade ambari to 240
# Perform RU to 2.5.0.0-1208

Deeper study shows that kerberos descriptor json in database ("artifact" table) 
still contains values and properties that are actual for 2.4 stack.
So the issue workflow should look like:
- Old stack version is installed
- Kerberos descriptor gets saved to database
- Security is enabled
- Stack upgrade is performed
- Keytab regeneration is performed, and it populates service config with 
obsolete property values

The issue happens on "Stack upgrade is performed" step. We never update 
kerberos descriptor json in database to correspond to a new stack.

>From nimbus.out
{code}Exception in thread "main" java.lang.ExceptionInInitializerError
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:270)
at clojure.lang.RT.classForName(RT.java:2154)
at clojure.lang.RT.classForName(RT.java:2163)
at clojure.lang.RT.loadClassForName(RT.java:2182)
at clojure.lang.RT.load(RT.java:436)
at clojure.lang.RT.load(RT.java:412)
at clojure.core$load$fn__5448.invoke(core.clj:5866)
at clojure.core$load.doInvoke(core.clj:5865)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at clojure.core$load_one.invoke(core.clj:5671)
at clojure.core$load_lib$fn__5397.invoke(core.clj:5711)
at clojure.core$load_lib.doInvoke(core.clj:5710)
at clojure.lang.RestFn.applyTo(RestFn.java:142)
at clojure.core$apply.invoke(core.clj:632)
at clojure.core$load_libs.doInvoke(core.clj:5749)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.core$apply.invoke(core.clj:632)
at clojure.core$require.doInvoke(core.clj:5832)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at 
org.apache.storm.daemon.nimbus$loading__5340__auto8560.invoke(nimbus.clj:16)
at org.apache.storm.daemon.nimbus__init.load(Unknown Source)
at org.apache.storm.daemon.nimbus__init.(Unknown Source)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:270)
at clojure.lang.RT.classForName(RT.java:2154)
at clojure.lang.RT.classForName(RT.java:2163)
at clojure.lang.RT.loadClassForName(RT.java:2182)
at clojure.lang.RT.load(RT.java:436)
at clojure.lang.RT.load(RT.java:412)
at clojure.core$load$fn__5448.invoke(core.clj:5866)
at clojure.core$load.doInvoke(core.clj:5865)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at clojure.lang.Var.invoke(Var.java:379)
at org.apache.storm.daemon.nimbus.(Unknown Source)
Caused by: java.lang.ClassNotFoundException: 
backtype.storm.security.auth.authorizer.SimpleACLAuthorizer
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:190)
at 
org.apache.storm.daemon.common$mk_authorization_handler.invoke(common.clj:412)
at org.apache.storm.ui.core__init.load(Unknown Source)
at org.apache.storm.ui.core__init.(Unknown Source)
... 35 more
{code}


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/StormUpgradeKerberosDescriptorConfig.java
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml
 65a7880 
  ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
bc8e9f9 
  
ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml
 98e5038 
  ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
e886301 

Diff: https://reviews.apache.org/r/51241/diff/


Testing
---

As of now, testing on live cluster


Thanks,

Dmitro Lisnichenko

Review Request 51242: Restify logsearch endpoints

2016-08-19 Thread Oliver Szabo 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51242/
---

Review request for Ambari, Dharmesh Makwana, Miklos Gergely, Robert Nettleton, 
and Sumit Mohanty.


Bugs: AMBARI-18214
https://issues.apache.org/jira/browse/AMBARI-18214


Repository: ambari


Description
---

- restify endpoints as possible
- rename dashboard to service
- there are still some endpoints what is not too resty, but that is the "vol 1" 
for refactoring. in some cases, some of the endpoints can be merged into one, 
and call a same method with just different params. for now, i kept them, i did 
not want to do any changes in the business code part. (after Miklos is ready 
with the refactoring, we can do some changes there too, and get rid some weird 
names in the API ... like verbs e.g.: /service/logs/aggregated -> it can be 
also in /service/logs endpoint and add aggregated as a query param etc.)


Diffs
-

  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/SolrDaoBase.java
 672507a 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/doc/DocConstants.java
 c1572b7 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/manager/AuditMgr.java
 d4f2986 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/manager/LogsMgr.java
 b03a643 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/rest/AuditLogsREST.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/rest/AuditREST.java
 5ed49fd 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/rest/DashboardREST.java
 0144edc 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/rest/LogFileREST.java
 d53cff9 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/rest/PublicREST.java
 af48acd 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/rest/ServiceLogsREST.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/rest/UserConfigREST.java
 4b1675f 
  ambari-logsearch/ambari-logsearch-portal/src/main/webapp/login.html 44f1aeb 
  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/collection_bases/VAuditLogListBase.js
 12f7c31 
  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/collection_bases/VEventHistoryListBase.js
 f6f720d 
  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/collection_bases/VGroupListBase.js
 a34aaa3 
  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/collection_bases/VLogLevelListBase.js
 59b5ae8 
  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/collection_bases/VLogListBase.js
 7b102d5 
  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/collection_bases/VNameValueListBase.js
 d59eaa2 
  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/collection_bases/VNodeListBase.js
 7c7dcf8 
  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/model_bases/VAuditLogBase.js
 1283875 
  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/model_bases/VCommonModelBase.js
 4723e3e 
  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/model_bases/VEventHistoryBase.js
 c237ade 
  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/model_bases/VGraphInfoBase.js
 a707629 
  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/model_bases/VLogLevelBase.js
 0384bc2 
  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/model_bases/VUserFilterBase.js
 35171aa 
  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/utils/ViewUtils.js
 331ffd6 
  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/views/audit/AuditAggregatedView.js
 c04aaf9 
  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/views/audit/AuditTabLayoutView.js
 0b570ac 
  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/views/dashboard/BubbleGraphTableLayoutView.js
 42b94d5 
  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/views/dashboard/ComponentListView.js
 abd3740 
  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/views/dashboard/ComponentsView.js
 66cc277 
  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/views/dashboard/DashboardView.js
 c3fd9c2 
  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/views/dashboard/GridTableLayoutView.js
 1cbdef8 
  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/views/dashboard/HostsView.js
 dd82130