date:20160523

Re: Review Request 47749: Warning for HAWQSTANDBY does not show up on Assign Masters page

2016-05-23 Thread Alexander Denissov


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47749/#review134514
---


Ship it!




Ship It!

- Alexander Denissov


On May 23, 2016, 11:44 p.m., Matt wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47749/
> ---
> 
> (Updated May 23, 2016, 11:44 p.m.)
> 
> 
> Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, and Lav 
> Jain.
> 
> 
> Bugs: AMBARI-16829
> https://issues.apache.org/jira/browse/AMBARI-16829
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Warning for HAWQSTANDBY does not show up on Assign Masters page
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py
>  c5813ad 
>   ambari-server/src/test/python/stacks/2.3/HAWQ/test_service_advisor.py 
> c33a223 
> 
> Diff: https://reviews.apache.org/r/47749/diff/
> 
> 
> Testing
> ---
> 
> Manually tested.
> 
> Added new test case.
> 
> ```
> test_createComponentLayoutRecommendations_hawq_1_Host 
> (test_stack_advisor.TestHDP23StackAdvisor) ... ok
> test_createComponentLayoutRecommendations_hawq_3_Hosts 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test that HAWQSTANDBY is recommended on a 3-node cluster ... ok
> test_createComponentLayoutRecommendations_hawqsegment_add_service_wizard_already_installed
>  (test_stack_advisor.TestHDP23StackAdvisor)
> Test that HAWQSEGMENT does not get recommended during Add Service Wizard, 
> when HAWQ has already been installed ... ok
> test_createComponentLayoutRecommendations_hawqsegment_add_service_wizard_to_be_installed
>  (test_stack_advisor.TestHDP23StackAdvisor)
> Test that HAWQSEGMENT gets recommended correctly during Add Service Wizard, 
> when HAWQ is selected for installation ... ok
> test_createComponentLayoutRecommendations_hawqsegment_cluster_install 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test that HAWQSEGMENT gets recommended correctly during Cluster Install 
> Wizard, when HAWQ is selected for installation ... ok
> test_createComponentLayoutRecommendations_no_hawq_3_Hosts 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test no failures when there are no HAWQ components ... ok
> test_createComponentLayoutRecommendations_pxf_add_service_wizard_already_installed
>  (test_stack_advisor.TestHDP23StackAdvisor)
> Test that PXF does not get recommended during Add Service Wizard, when PXF 
> has already been installed ... ok
> test_createComponentLayoutRecommendations_pxf_add_service_wizard_to_be_installed
>  (test_stack_advisor.TestHDP23StackAdvisor)
> Test that PXF gets recommended correctly during Add Service Wizard, when PXF 
> is selected for installation ... ok
> test_createComponentLayoutRecommendations_pxf_cluster_install 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test that PXF gets recommended correctly during Cluster Install Wizard, when 
> PXF is selected for installation ... ok
> test_getComponentLayoutValidations_hawq_3_Hosts 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test layout validations for HAWQ components on a 3-node cluster ... ok
> test_getComponentLayoutValidations_hawqsegment_not_co_located_with_datanode 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test validation warning for HAWQ segment not colocated with DATANODE ... ok
> test_getComponentLayoutValidations_nohawq_3_Hosts 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test no failures when there are no HAWQ components on a 3-node cluster ... ok
> test_getComponentLayoutValidations_pxf_co_located_with_nn_and_dn 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test NO warning is generated when PXF is co-located with NAMENODE and 
> DATANODE ... ok
> test_getComponentLayoutValidations_pxf_not_co_located_with_dn 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test warning is generated when PXF is not co-located with NAMENODE or 
> DATANODE ... ok
> test_getComponentLayoutValidations_pxf_not_co_located_with_nn 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test warning is generated when PXF is not co-located with NAMENODE ... ok
> test_getComponentLayoutValidations_pxf_not_co_located_with_nn_or_dn 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test warning is generated when PXF is not co-located with NAMENODE or 
> DATANODE ... ok
> test_getComponentLayoutValidations_sparkts_no_hive 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test SparkTS is picked when Hive is not installed ... ok
> test_getComponentLayoutValidations_sparkts_with_hive 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test SparkTS is picked when Hive is installed ... ok
> test_noRiskyDictLookups (test_stack_advisor.TestHDP23StackAdvisor) ... ok
> test_recommendFalconConfigurations (test_stack_advisor.TestHDP23StackAdvisor) 
> ...

Re: Review Request 47747: Support round-robin scheduling with failover for Sinks with distributed collector

2016-05-23 Thread Aravindan Vijayan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47747/#review134513
---


Ship it!




Will test this out and refine further.

- Aravindan Vijayan


On May 23, 2016, 11:25 p.m., Sid Wagle wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47747/
> ---
> 
> (Updated May 23, 2016, 11:25 p.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Dmytro Sen, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-16828
> https://issues.apache.org/jira/browse/AMBARI-16828
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> - Initial set of collectors is configured in the configuration files
> - Find available collectors by connecting to zookeeper thereafter 
> - Remember available collectors, refresh this information only when one 
> collector cannot be reached with a very low frequency of checks, example: 
> random interval between 10-12 minutes, check if a new collector is available. 
> Set a low client side zk timeout.
> - Round robin the write between the collector choosing the first one at random
> - If a write timed out, choose the next available collector, remember the 
> attempts with the first one
> - Set a configurable attempt count for failed connector (default = 3), after 
> which the failed connector is no longer in the available collectors list. 
> - The next retry will be triggered after refresh with zookeeper is successful
> - If no failed collectors available, zk refresh interval should be chosen 
> randomly between 1-2 minutes.
> 
> 
> Diffs
> -
> 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/LogFeederAMSClient.java
>  7eed376 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/MetricsMgr.java
>  2152d14 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/solr/metrics/SolrAmsClient.java
>  cdeb63d 
>   ambari-metrics/ambari-metrics-common/pom.xml 5a388dd 
>   
> ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/AbstractTimelineMetricsSink.java
>  5a716df 
>   
> ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/MetricsSinkInitializationException.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/availability/MetricCollectorHAHelper.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/availability/MetricCollectorUnavailableException.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/availability/MetricSinkWriteShardHostnameHashingStrategy.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/availability/MetricSinkWriteShardStrategy.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-common/src/test/java/org/apache/hadoop/metrics2/sink/timeline/availability/MetricCollectorHATest.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-common/src/test/java/org/apache/hadoop/metrics2/sink/timeline/availability/ShardingStrategyTest.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-common/src/test/java/org/apache/hadoop/metrics2/sink/timeline/cache/HandleConnectExceptionTest.java
>  a192802 
>   
> ambari-metrics/ambari-metrics-flume-sink/src/main/java/org/apache/hadoop/metrics2/sink/flume/FlumeTimelineMetricsSink.java
>  3040c48 
>   
> ambari-metrics/ambari-metrics-hadoop-sink/src/main/java/org/apache/hadoop/metrics2/sink/timeline/HadoopTimelineMetricsSink.java
>  8966978 
>   
> ambari-metrics/ambari-metrics-hadoop-sink/src/test/java/org/apache/hadoop/metrics2/sink/timeline/HadoopTimelineMetricsSinkTest.java
>  4a5abcc 
>   
> ambari-metrics/ambari-metrics-kafka-sink/src/main/java/org/apache/hadoop/metrics2/sink/kafka/KafkaTimelineMetricsReporter.java
>  d6d251c 
>   
> ambari-metrics/ambari-metrics-kafka-sink/src/test/java/org/apache/hadoop/metrics2/sink/kafka/KafkaTimelineMetricsReporterTest.java
>  e0adb4b 
>   
> ambari-metrics/ambari-metrics-storm-sink/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsReporter.java
>  ab5f1e4 
>   
> ambari-metrics/ambari-metrics-storm-sink/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSink.java
>  91c63b9 
>   
> ambari-metrics/ambari-metrics-storm-sink/src/test/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSinkTest.java
>  3f139da 
>   
>

Re: Review Request 47596: HiveServer interactive - incorrect default memory value

2016-05-23 Thread Swapan Shridhar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47596/#review134511
---




ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py (line 
221)


Can you explain how moving this line up helps ? Thx.


- Swapan Shridhar


On May 23, 2016, 8:20 a.m., Daniel Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47596/
> ---
> 
> (Updated May 23, 2016, 8:20 a.m.)
> 
> 
> Review request for Ambari, Laszlo Puskas, Oliver Szabo, Sandor Magyari, Sumit 
> Mohanty, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-16776
> https://issues.apache.org/jira/browse/AMBARI-16776
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Default value of hive.llap.daemon.yarn.container.mb is not aligned to 
> yarn.scheduler.minimum-allocation-mb
> If the former one is less than the latter one, then hive server interactive 
> wont start. So the default values must be in a proper relation.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py
>  9f5d799 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-interactive-site.xml
>  db5f616 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
> 11aac72 
> 
> Diff: https://reviews.apache.org/r/47596/diff/
> 
> 
> Testing
> ---
> 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 2:15.746s
> [INFO] Finished at: Thu May 19 18:52:43 CEST 2016
> [INFO] Final Memory: 58M/1120M
> [INFO] 
> 
> 
> 
> Thanks,
> 
> Daniel Gergely
> 
>

Re: Review Request 47757: Desired Configuration Cache Expiration Caused 10, 000's of Database Hits In Large Deployments

2016-05-23 Thread Jonathan Hurley


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47757/#review134495
---




ambari-server/src/main/java/org/apache/ambari/annotations/TransactionalLock.java
 (lines 70 - 73)


The stale config cache can suffer the same transactional/cache-update 
problem as the HRC. We should be locking in some places to ensure that an 
invalidation doesn't re-populate a stale value before the transaction commits.



ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
 (line 1168)


One of the many areas in which we can now fetch these once ahead of time 
and pass them around.



ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java 


This was a bad idea; force a database hit every minute for every SCH ... 
The problem was that stale configs were not being reported correctly. It seems 
to be an issue with the cache invalidation while re-populating inside a 
transaction. Anway - bad idea - we cannot do this.



ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 
(lines 190 - 193)


Method contract allows for null for those times we are lazy. Let's hope 
we're not lazy in a loop.



ambari-server/src/main/java/org/apache/ambari/server/state/ServiceComponentHost.java
 (line 181)


AMC loops over every SCH ... pass in the configs here as well so that they 
are only retrieved once.



ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
 (line 2382)


This should always be correct - there should be no reason to bypass it.


- Jonathan Hurley


On May 23, 2016, 10:38 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47757/
> ---
> 
> (Updated May 23, 2016, 10:38 p.m.)
> 
> 
> Review request for Ambari and Nate Cole.
> 
> 
> Bugs: AMBARI-16830
> https://issues.apache.org/jira/browse/AMBARI-16830
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> The stale configuration cache is set to expire every 1 minute. When it 
> expires, several areas of code invoke the same code path to rebuild the 
> cache, which essentially loops over all SCH's.
> 
> In a large deployment, that could be 10,000 SCHs every minute. At an average 
> of 10ms per database round trip, that's a lot more than 1 minute. Increasing 
> the cache invalidation doesn't really help since editing a configuration will 
> invalidate the cache anyway, causing the UI to hang.
> 
> Instead, we should be relying on the JPA EM relationships. At the same time, 
> there are several areas (like cluster health report) where the database/cache 
> hit can be done ahead of time once and passed in for every SCH.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/annotations/TransactionalLock.java
>  cd961ba 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/HeartbeatMonitor.java
>  a902a2c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  dd3d098 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostResourceProvider.java
>  8471825 
>   ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java 
> d9df8b4 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 
> 854dd0b 
>   ambari-server/src/main/java/org/apache/ambari/server/state/Host.java 
> e9e5ea1 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/ServiceComponentHost.java
>  4866148 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
>  6cf9933 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/host/HostImpl.java 
> e3665bf 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java
>  feae6e5 
>   
> ambari-server/src/test/java/org/apache/ambari/server/configuration/ConfigurationTest.java
>  7e8c144 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerImplTest.java
>  1a7ff67 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/RefreshYarnCapacitySchedulerReleaseConfigTest.java
>  179f658 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/HostResourceProviderTest.java
>  ca332fd 
>   
>

Review Request 47757: Desired Configuration Cache Expiration Caused 10, 000's of Database Hits In Large Deployments

2016-05-23 Thread Jonathan Hurley


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47757/
---

Review request for Ambari and Nate Cole.


Bugs: AMBARI-16830
https://issues.apache.org/jira/browse/AMBARI-16830


Repository: ambari


Description
---

The stale configuration cache is set to expire every 1 minute. When it expires, 
several areas of code invoke the same code path to rebuild the cache, which 
essentially loops over all SCH's.

In a large deployment, that could be 10,000 SCHs every minute. At an average of 
10ms per database round trip, that's a lot more than 1 minute. Increasing the 
cache invalidation doesn't really help since editing a configuration will 
invalidate the cache anyway, causing the UI to hang.

Instead, we should be relying on the JPA EM relationships. At the same time, 
there are several areas (like cluster health report) where the database/cache 
hit can be done ahead of time once and passed in for every SCH.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/annotations/TransactionalLock.java
 cd961ba 
  
ambari-server/src/main/java/org/apache/ambari/server/agent/HeartbeatMonitor.java
 a902a2c 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
 dd3d098 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostResourceProvider.java
 8471825 
  ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java 
d9df8b4 
  ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 
854dd0b 
  ambari-server/src/main/java/org/apache/ambari/server/state/Host.java e9e5ea1 
  
ambari-server/src/main/java/org/apache/ambari/server/state/ServiceComponentHost.java
 4866148 
  
ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
 6cf9933 
  ambari-server/src/main/java/org/apache/ambari/server/state/host/HostImpl.java 
e3665bf 
  
ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java
 feae6e5 
  
ambari-server/src/test/java/org/apache/ambari/server/configuration/ConfigurationTest.java
 7e8c144 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerImplTest.java
 1a7ff67 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/RefreshYarnCapacitySchedulerReleaseConfigTest.java
 179f658 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/HostResourceProviderTest.java
 ca332fd 
  
ambari-server/src/test/java/org/apache/ambari/server/state/ConfigHelperTest.java
 15bcc53 
  
ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClusterDeadlockTest.java
 0a3286c 
  
ambari-server/src/test/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostTest.java
 57a7391 

Diff: https://reviews.apache.org/r/47757/diff/


Testing
---


Thanks,

Jonathan Hurley

Review Request 47751: AMBARI-16831 - Atlas metadaserver doesn't show up in warning popup while deleting a host

2016-05-23 Thread Richard Zang


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47751/
---

Review request for Ambari, Jaimin Jetly and Zhe (Joe) Wang.


Bugs: AMBARI-16831
https://issues.apache.org/jira/browse/AMBARI-16831


Repository: ambari


Description
---

Include deletable master components.


Diffs
-

  ambari-web/app/controllers/main/host/details.js b16ef46 
  ambari-web/app/messages.js fe5a301 

Diff: https://reviews.apache.org/r/47751/diff/


Testing
---

Manually tested on live cluster.
All unit tests passed.
  27834 tests complete (33 seconds)
  154 tests pending


Thanks,

Richard Zang

Re: Review Request 47749: Warning for HAWQSTANDBY does not show up on Assign Masters page

2016-05-23 Thread Lav Jain


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47749/#review134490
---


Ship it!




Ship It!

- Lav Jain


On May 23, 2016, 11:44 p.m., Matt wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47749/
> ---
> 
> (Updated May 23, 2016, 11:44 p.m.)
> 
> 
> Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, and Lav 
> Jain.
> 
> 
> Bugs: AMBARI-16829
> https://issues.apache.org/jira/browse/AMBARI-16829
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Warning for HAWQSTANDBY does not show up on Assign Masters page
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py
>  c5813ad 
>   ambari-server/src/test/python/stacks/2.3/HAWQ/test_service_advisor.py 
> c33a223 
> 
> Diff: https://reviews.apache.org/r/47749/diff/
> 
> 
> Testing
> ---
> 
> Manually tested.
> 
> Added new test case.
> 
> ```
> test_createComponentLayoutRecommendations_hawq_1_Host 
> (test_stack_advisor.TestHDP23StackAdvisor) ... ok
> test_createComponentLayoutRecommendations_hawq_3_Hosts 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test that HAWQSTANDBY is recommended on a 3-node cluster ... ok
> test_createComponentLayoutRecommendations_hawqsegment_add_service_wizard_already_installed
>  (test_stack_advisor.TestHDP23StackAdvisor)
> Test that HAWQSEGMENT does not get recommended during Add Service Wizard, 
> when HAWQ has already been installed ... ok
> test_createComponentLayoutRecommendations_hawqsegment_add_service_wizard_to_be_installed
>  (test_stack_advisor.TestHDP23StackAdvisor)
> Test that HAWQSEGMENT gets recommended correctly during Add Service Wizard, 
> when HAWQ is selected for installation ... ok
> test_createComponentLayoutRecommendations_hawqsegment_cluster_install 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test that HAWQSEGMENT gets recommended correctly during Cluster Install 
> Wizard, when HAWQ is selected for installation ... ok
> test_createComponentLayoutRecommendations_no_hawq_3_Hosts 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test no failures when there are no HAWQ components ... ok
> test_createComponentLayoutRecommendations_pxf_add_service_wizard_already_installed
>  (test_stack_advisor.TestHDP23StackAdvisor)
> Test that PXF does not get recommended during Add Service Wizard, when PXF 
> has already been installed ... ok
> test_createComponentLayoutRecommendations_pxf_add_service_wizard_to_be_installed
>  (test_stack_advisor.TestHDP23StackAdvisor)
> Test that PXF gets recommended correctly during Add Service Wizard, when PXF 
> is selected for installation ... ok
> test_createComponentLayoutRecommendations_pxf_cluster_install 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test that PXF gets recommended correctly during Cluster Install Wizard, when 
> PXF is selected for installation ... ok
> test_getComponentLayoutValidations_hawq_3_Hosts 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test layout validations for HAWQ components on a 3-node cluster ... ok
> test_getComponentLayoutValidations_hawqsegment_not_co_located_with_datanode 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test validation warning for HAWQ segment not colocated with DATANODE ... ok
> test_getComponentLayoutValidations_nohawq_3_Hosts 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test no failures when there are no HAWQ components on a 3-node cluster ... ok
> test_getComponentLayoutValidations_pxf_co_located_with_nn_and_dn 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test NO warning is generated when PXF is co-located with NAMENODE and 
> DATANODE ... ok
> test_getComponentLayoutValidations_pxf_not_co_located_with_dn 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test warning is generated when PXF is not co-located with NAMENODE or 
> DATANODE ... ok
> test_getComponentLayoutValidations_pxf_not_co_located_with_nn 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test warning is generated when PXF is not co-located with NAMENODE ... ok
> test_getComponentLayoutValidations_pxf_not_co_located_with_nn_or_dn 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test warning is generated when PXF is not co-located with NAMENODE or 
> DATANODE ... ok
> test_getComponentLayoutValidations_sparkts_no_hive 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test SparkTS is picked when Hive is not installed ... ok
> test_getComponentLayoutValidations_sparkts_with_hive 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test SparkTS is picked when Hive is installed ... ok
> test_noRiskyDictLookups (test_stack_advisor.TestHDP23StackAdvisor) ... ok
> test_recommendFalconConfigurations (test_stack_advisor.TestHDP23StackAdvisor) 
> ... ok
>

Re: Review Request 47746: Update Moment.js to latest stable version 2.13.0

2016-05-23 Thread Jaimin Jetly


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47746/#review134482
---



Sangeeta Ravindran, Lets please add minified version of moment.js for 2.13.0 
version.

- Jaimin Jetly


On May 23, 2016, 11:50 p.m., Sangeeta Ravindran wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47746/
> ---
> 
> (Updated May 23, 2016, 11:50 p.m.)
> 
> 
> Review request for Ambari, Jaimin Jetly and Yusaku Sako.
> 
> 
> Bugs: ambari-16017
> https://issues.apache.org/jira/browse/ambari-16017
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> The latest stable version of Moment.js is 2.13.0.
> This task is for updating the version of Moment.js used in
> 
> contrib/views/slider/src/main/resources/ui/vendor/scripts/common/
> ambari-web/vendor/scripts
> 
> 
> Diffs
> -
> 
>   ambari-web/vendor/scripts/moment.js fe0e19c 
>   contrib/views/slider/src/main/resources/ui/vendor/scripts/common/moment.js 
> b40374b 
> 
> Diff: https://reviews.apache.org/r/47746/diff/
> 
> 
> Testing
> ---
> 
> Manual testing - setting timezone in User Settings.
> Slider view - application start time.
> 
> Tests ran clean for ambari-web 
> 
> 27834 tests complete (42 seconds)
>   154 tests pending
> 
> Tests ran clean for contrib/views/slider
> Took 9506ms to run 345 tests. 345 passed, 0 failed.
> 
> 
> Thanks,
> 
> Sangeeta Ravindran
> 
>

Re: Review Request 47452: Do not use implicit routing by default in solr-client

2016-05-23 Thread Don Bosco Durai


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47452/#review134480
---


Ship it!




Ship It!

- Don Bosco Durai


On May 17, 2016, 1:29 p.m., Oliver Szabo wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47452/
> ---
> 
> (Updated May 17, 2016, 1:29 p.m.)
> 
> 
> Review request for Ambari, Don Bosco Durai, Oliver Szabo, Robert Nettleton, 
> and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-16701
> https://issues.apache.org/jira/browse/AMBARI-16701
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> By default ambari should not use implicit routing for creating collections
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/solr_cloud_util.py
>  d661288 
>   
> ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/AmbariSolrCloudCLI.java
>  21e7fee 
>   
> ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/commands/CreateCollectionCommand.java
>  c888269 
> 
> Diff: https://reviews.apache.org/r/47452/diff/
> 
> 
> Testing
> ---
> 
> ambari logsearch solr-client tests:
> Tests run: 4, Failures: 0, Errors: 0, Skipped: 0
> 
> 
> Thanks,
> 
> Oliver Szabo
> 
>

Re: Review Request 47452: Do not use implicit routing by default in solr-client

2016-05-23 Thread Oliver Szabo



> On May 23, 2016, 11:24 p.m., Don Bosco Durai wrote:
> > ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/commands/CreateCollectionCommand.java,
> >  line 54
> > 
> >
> > During Create collection, shards is a valid value. Only after the 
> > collection is created, you can't add shards to SolrCloud using compositeId

sorry, the comment is below for this one


- Oliver


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47452/#review134474
---


On May 17, 2016, 1:29 p.m., Oliver Szabo wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47452/
> ---
> 
> (Updated May 17, 2016, 1:29 p.m.)
> 
> 
> Review request for Ambari, Don Bosco Durai, Oliver Szabo, Robert Nettleton, 
> and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-16701
> https://issues.apache.org/jira/browse/AMBARI-16701
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> By default ambari should not use implicit routing for creating collections
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/solr_cloud_util.py
>  d661288 
>   
> ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/AmbariSolrCloudCLI.java
>  21e7fee 
>   
> ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/commands/CreateCollectionCommand.java
>  c888269 
> 
> Diff: https://reviews.apache.org/r/47452/diff/
> 
> 
> Testing
> ---
> 
> ambari logsearch solr-client tests:
> Tests run: 4, Failures: 0, Errors: 0, Skipped: 0
> 
> 
> Thanks,
> 
> Oliver Szabo
> 
>

Re: Review Request 47452: Do not use implicit routing by default in solr-client

2016-05-23 Thread Oliver Szabo


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47452/#review134478
---




ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/commands/CreateCollectionCommand.java
 (line 53)


It's like that you described. but you cam switch that off, by default its 
used.


- Oliver Szabo


On May 17, 2016, 1:29 p.m., Oliver Szabo wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47452/
> ---
> 
> (Updated May 17, 2016, 1:29 p.m.)
> 
> 
> Review request for Ambari, Don Bosco Durai, Oliver Szabo, Robert Nettleton, 
> and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-16701
> https://issues.apache.org/jira/browse/AMBARI-16701
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> By default ambari should not use implicit routing for creating collections
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/solr_cloud_util.py
>  d661288 
>   
> ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/AmbariSolrCloudCLI.java
>  21e7fee 
>   
> ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/commands/CreateCollectionCommand.java
>  c888269 
> 
> Diff: https://reviews.apache.org/r/47452/diff/
> 
> 
> Testing
> ---
> 
> ambari logsearch solr-client tests:
> Tests run: 4, Failures: 0, Errors: 0, Skipped: 0
> 
> 
> Thanks,
> 
> Oliver Szabo
> 
>

Review Request 47746: Update Moment.js to latest stable version 2.13.0

2016-05-23 Thread Sangeeta Ravindran


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47746/
---

Review request for Ambari, Jaimin Jetly and Yusaku Sako.


Bugs: ambari-16017
https://issues.apache.org/jira/browse/ambari-16017


Repository: ambari


Description
---

The latest stable version of Moment.js is 2.13.0.
This task is for updating the version of Moment.js used in

contrib/views/slider/src/main/resources/ui/vendor/scripts/common/
ambari-web/vendor/scripts


Diffs
-

  ambari-web/vendor/scripts/moment.js fe0e19c 
  contrib/views/slider/src/main/resources/ui/vendor/scripts/common/moment.js 
b40374b 

Diff: https://reviews.apache.org/r/47746/diff/


Testing
---

Manual testing - setting timezone in User Settings.
Slider view - application start time.

Tests ran clean for ambari-web 

27834 tests complete (42 seconds)
  154 tests pending

Tests ran clean for contrib/views/slider
Took 9506ms to run 345 tests. 345 passed, 0 failed.


Thanks,

Sangeeta Ravindran

Review Request 47747: Support round-robin scheduling with failover for Sinks with distributed collector

2016-05-23 Thread Sid Wagle


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47747/
---

Review request for Ambari, Aravindan Vijayan, Dmytro Sen, and Sumit Mohanty.


Bugs: AMBARI-16828
https://issues.apache.org/jira/browse/AMBARI-16828


Repository: ambari


Description
---

- Initial set of collectors is configured in the configuration files
- Find available collectors by connecting to zookeeper thereafter 
- Remember available collectors, refresh this information only when one 
collector cannot be reached with a very low frequency of checks, example: 
random interval between 10-12 minutes, check if a new collector is available. 
Set a low client side zk timeout.
- Round robin the write between the collector choosing the first one at random
- If a write timed out, choose the next available collector, remember the 
attempts with the first one
- Set a configurable attempt count for failed connector (default = 3), after 
which the failed connector is no longer in the available collectors list. 
- The next retry will be triggered after refresh with zookeeper is successful
- If no failed collectors available, zk refresh interval should be chosen 
randomly between 1-2 minutes.


Diffs
-

  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/LogFeederAMSClient.java
 7eed376 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/MetricsMgr.java
 2152d14 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/solr/metrics/SolrAmsClient.java
 cdeb63d 
  ambari-metrics/ambari-metrics-common/pom.xml 5a388dd 
  
ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/AbstractTimelineMetricsSink.java
 5a716df 
  
ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/MetricsSinkInitializationException.java
 PRE-CREATION 
  
ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/availability/MetricCollectorHAHelper.java
 PRE-CREATION 
  
ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/availability/MetricCollectorUnavailableException.java
 PRE-CREATION 
  
ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/availability/MetricSinkWriteShardHostnameHashingStrategy.java
 PRE-CREATION 
  
ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/availability/MetricSinkWriteShardStrategy.java
 PRE-CREATION 
  
ambari-metrics/ambari-metrics-common/src/test/java/org/apache/hadoop/metrics2/sink/timeline/availability/MetricCollectorHATest.java
 PRE-CREATION 
  
ambari-metrics/ambari-metrics-common/src/test/java/org/apache/hadoop/metrics2/sink/timeline/availability/ShardingStrategyTest.java
 PRE-CREATION 
  
ambari-metrics/ambari-metrics-common/src/test/java/org/apache/hadoop/metrics2/sink/timeline/cache/HandleConnectExceptionTest.java
 a192802 
  
ambari-metrics/ambari-metrics-flume-sink/src/main/java/org/apache/hadoop/metrics2/sink/flume/FlumeTimelineMetricsSink.java
 3040c48 
  
ambari-metrics/ambari-metrics-hadoop-sink/src/main/java/org/apache/hadoop/metrics2/sink/timeline/HadoopTimelineMetricsSink.java
 8966978 
  
ambari-metrics/ambari-metrics-hadoop-sink/src/test/java/org/apache/hadoop/metrics2/sink/timeline/HadoopTimelineMetricsSinkTest.java
 4a5abcc 
  
ambari-metrics/ambari-metrics-kafka-sink/src/main/java/org/apache/hadoop/metrics2/sink/kafka/KafkaTimelineMetricsReporter.java
 d6d251c 
  
ambari-metrics/ambari-metrics-kafka-sink/src/test/java/org/apache/hadoop/metrics2/sink/kafka/KafkaTimelineMetricsReporterTest.java
 e0adb4b 
  
ambari-metrics/ambari-metrics-storm-sink/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsReporter.java
 ab5f1e4 
  
ambari-metrics/ambari-metrics-storm-sink/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSink.java
 91c63b9 
  
ambari-metrics/ambari-metrics-storm-sink/src/test/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSinkTest.java
 3f139da 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java
 fb83a6b 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/AbstractTimelineAggregator.java
 ae87cf1 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricAggregatorFactory.java
 4c44f9e

Re: Review Request 47452: Do not use implicit routing by default in solr-client

2016-05-23 Thread Don Bosco Durai


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47452/#review134474
---


Fix it, then Ship it!




Ship it after reviewing feedback. Thanks


ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/commands/CreateCollectionCommand.java
 (line 53)


During Create collection, shards is a valid value. Only after the 
collection is created, you can't add shards to SolrCloud using compositeId


- Don Bosco Durai


On May 17, 2016, 1:29 p.m., Oliver Szabo wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47452/
> ---
> 
> (Updated May 17, 2016, 1:29 p.m.)
> 
> 
> Review request for Ambari, Don Bosco Durai, Oliver Szabo, Robert Nettleton, 
> and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-16701
> https://issues.apache.org/jira/browse/AMBARI-16701
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> By default ambari should not use implicit routing for creating collections
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/solr_cloud_util.py
>  d661288 
>   
> ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/AmbariSolrCloudCLI.java
>  21e7fee 
>   
> ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/commands/CreateCollectionCommand.java
>  c888269 
> 
> Diff: https://reviews.apache.org/r/47452/diff/
> 
> 
> Testing
> ---
> 
> ambari logsearch solr-client tests:
> Tests run: 4, Failures: 0, Errors: 0, Skipped: 0
> 
> 
> Thanks,
> 
> Oliver Szabo
> 
>

Re: Review Request 47739: AMBARI-16826: Unit Test failure: ComponentResourceProviderTest

2016-05-23 Thread Sumit Mohanty


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47739/#review134447
---


Ship it!




Ship It!

- Sumit Mohanty


On May 23, 2016, 9:44 p.m., Nahappan Somasundaram wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47739/
> ---
> 
> (Updated May 23, 2016, 9:44 p.m.)
> 
> 
> Review request for Ambari, Sumit Mohanty and Sid Wagle.
> 
> 
> Bugs: AMBARI-16826
> https://issues.apache.org/jira/browse/AMBARI-16826
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> AMBARI-16826: Unit Test failure: ComponentResourceProviderTest
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ComponentResourceProviderTest.java
>  7ec6e668aeec7a6ee667dd908ee1da7f33202fd1 
> 
> Diff: https://reviews.apache.org/r/47739/diff/
> 
> 
> Testing
> ---
> 
> ** 1. mvn clean install **
> 
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Ambari Main ... SUCCESS [10.520s]
> [INFO] Apache Ambari Project POM . SUCCESS [0.039s]
> [INFO] Ambari Web  SUCCESS [27.546s]
> [INFO] Ambari Views .. SUCCESS [1.447s]
> [INFO] Ambari Admin View . SUCCESS [9.824s]
> [INFO] ambari-metrics  SUCCESS [0.445s]
> [INFO] Ambari Metrics Common . SUCCESS [0.670s]
> [INFO] Ambari Metrics Hadoop Sink  SUCCESS [1.451s]
> [INFO] Ambari Metrics Flume Sink . SUCCESS [0.776s]
> [INFO] Ambari Metrics Kafka Sink . SUCCESS [0.756s]
> [INFO] Ambari Metrics Storm Sink . SUCCESS [1.828s]
> [INFO] Ambari Metrics Collector .. SUCCESS [6.627s]
> [INFO] Ambari Metrics Monitor  SUCCESS [1.771s]
> [INFO] Ambari Metrics Grafana  SUCCESS [0.897s]
> [INFO] Ambari Metrics Assembly ... SUCCESS [1:24.049s]
> [INFO] Ambari Server . SUCCESS [2:39.615s]
> [INFO] Ambari Functional Tests ... SUCCESS [2.063s]
> [INFO] Ambari Agent .. SUCCESS [25.139s]
> [INFO] Ambari Client . SUCCESS [0.048s]
> [INFO] Ambari Python Client .. SUCCESS [0.978s]
> [INFO] Ambari Groovy Client .. SUCCESS [2.160s]
> [INFO] Ambari Shell .. SUCCESS [0.046s]
> [INFO] Ambari Python Shell ... SUCCESS [0.686s]
> [INFO] Ambari Groovy Shell ... SUCCESS [0.914s]
> [INFO] ambari-logsearch .. SUCCESS [0.218s]
> [INFO] Ambari Logsearch Appender . SUCCESS [0.210s]
> [INFO] Ambari Logsearch Solr Client .. SUCCESS [1.137s]
> [INFO] Ambari Logsearch Portal ... SUCCESS [6.887s]
> [INFO] Ambari Logsearch Log Feeder ... SUCCESS [2.231s]
> [INFO] Ambari Logsearch Assembly . SUCCESS [0.094s]
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 5:53.142s
> [INFO] Finished at: Mon May 23 13:36:22 PDT 2016
> [INFO] Final Memory: 292M/1222M
> [INFO] 
> 
> 
> ** 2. mvn test -DskipPythonTests **
> 
> Results :
> 
> Tests run: 4367, Failures: 0, Errors: 0, Skipped: 34
> 
> [INFO] Rat check: Summary of files. Unapproved: 0 unknown: 0 generated: 0 
> approved: 4708 licence.
> [INFO]
> [INFO] --- exec-maven-plugin:1.2.1:exec (python-test) @ ambari-server ---
> [INFO] skipping execute as per configuraion
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 1:04:25.638s
> [INFO] Finished at: Mon May 23 14:41:50 PDT 2016
> [INFO] Final Memory: 35M/582M
> [INFO] 
> 
> 
> 
> Thanks,
> 
> Nahappan Somasundaram
> 
>

Review Request 47739: AMBARI-16826: Unit Test failure: ComponentResourceProviderTest

2016-05-23 Thread Nahappan Somasundaram


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47739/
---

Review request for Ambari, Sumit Mohanty and Sid Wagle.


Bugs: AMBARI-16826
https://issues.apache.org/jira/browse/AMBARI-16826


Repository: ambari


Description
---

AMBARI-16826: Unit Test failure: ComponentResourceProviderTest


Diffs
-

  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ComponentResourceProviderTest.java
 7ec6e668aeec7a6ee667dd908ee1da7f33202fd1 

Diff: https://reviews.apache.org/r/47739/diff/


Testing
---

** 1. mvn clean install **

[INFO] 
[INFO] Reactor Summary:
[INFO]
[INFO] Ambari Main ... SUCCESS [10.520s]
[INFO] Apache Ambari Project POM . SUCCESS [0.039s]
[INFO] Ambari Web  SUCCESS [27.546s]
[INFO] Ambari Views .. SUCCESS [1.447s]
[INFO] Ambari Admin View . SUCCESS [9.824s]
[INFO] ambari-metrics  SUCCESS [0.445s]
[INFO] Ambari Metrics Common . SUCCESS [0.670s]
[INFO] Ambari Metrics Hadoop Sink  SUCCESS [1.451s]
[INFO] Ambari Metrics Flume Sink . SUCCESS [0.776s]
[INFO] Ambari Metrics Kafka Sink . SUCCESS [0.756s]
[INFO] Ambari Metrics Storm Sink . SUCCESS [1.828s]
[INFO] Ambari Metrics Collector .. SUCCESS [6.627s]
[INFO] Ambari Metrics Monitor  SUCCESS [1.771s]
[INFO] Ambari Metrics Grafana  SUCCESS [0.897s]
[INFO] Ambari Metrics Assembly ... SUCCESS [1:24.049s]
[INFO] Ambari Server . SUCCESS [2:39.615s]
[INFO] Ambari Functional Tests ... SUCCESS [2.063s]
[INFO] Ambari Agent .. SUCCESS [25.139s]
[INFO] Ambari Client . SUCCESS [0.048s]
[INFO] Ambari Python Client .. SUCCESS [0.978s]
[INFO] Ambari Groovy Client .. SUCCESS [2.160s]
[INFO] Ambari Shell .. SUCCESS [0.046s]
[INFO] Ambari Python Shell ... SUCCESS [0.686s]
[INFO] Ambari Groovy Shell ... SUCCESS [0.914s]
[INFO] ambari-logsearch .. SUCCESS [0.218s]
[INFO] Ambari Logsearch Appender . SUCCESS [0.210s]
[INFO] Ambari Logsearch Solr Client .. SUCCESS [1.137s]
[INFO] Ambari Logsearch Portal ... SUCCESS [6.887s]
[INFO] Ambari Logsearch Log Feeder ... SUCCESS [2.231s]
[INFO] Ambari Logsearch Assembly . SUCCESS [0.094s]
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 5:53.142s
[INFO] Finished at: Mon May 23 13:36:22 PDT 2016
[INFO] Final Memory: 292M/1222M
[INFO] 

** 2. mvn test -DskipPythonTests **

Results :

Tests run: 4367, Failures: 0, Errors: 0, Skipped: 34

[INFO] Rat check: Summary of files. Unapproved: 0 unknown: 0 generated: 0 
approved: 4708 licence.
[INFO]
[INFO] --- exec-maven-plugin:1.2.1:exec (python-test) @ ambari-server ---
[INFO] skipping execute as per configuraion
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 1:04:25.638s
[INFO] Finished at: Mon May 23 14:41:50 PDT 2016
[INFO] Final Memory: 35M/582M
[INFO] 


Thanks,

Nahappan Somasundaram

Re: Review Request 46661: Put HBase master UI into readonly mode for secure deployment for HDP 2.5

2016-05-23 Thread Robert Levas


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46661/#review134441
---


Ship it!




Ship It!

- Robert Levas


On May 23, 2016, 5:08 p.m., Ajit Kumar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46661/
> ---
> 
> (Updated May 23, 2016, 5:08 p.m.)
> 
> 
> Review request for Ambari, Robert Levas, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-16164
> https://issues.apache.org/jira/browse/AMBARI-16164
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Currently in secure deployment, user can request compaction / splitting 
> through hbase master UI.
> This potentially exposes vulnerability to various attacks.
> There is config parameter, hbase.master.ui.readonly, with default value of 
> false.
> In secure deployment, Master UI should be put to readonly mode (setting the 
> above parameter to true).
> Admin can always request compaction / splitting through hbase shell.
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
> 689e1fd 
> 
> Diff: https://reviews.apache.org/r/46661/diff/
> 
> 
> Testing
> ---
> 
> Manual testing, sceenshot attached (after kerberizing cluster).
> 
> 
> File Attachments
> 
> 
> hbase-site.png
>   
> https://reviews.apache.org/media/uploaded/files/2016/05/11/75e8d2ae-99a4-4e8d-8a69-52cd974734fb__hbase-site.png
> 
> 
> Thanks,
> 
> Ajit Kumar
> 
>

Re: Review Request 46661: Put HBase master UI into readonly mode for secure deployment for HDP 2.5

2016-05-23 Thread Ajit Kumar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46661/
---

(Updated May 23, 2016, 9:08 p.m.)


Review request for Ambari, Robert Levas, Sumit Mohanty, and Sid Wagle.


Changes
---

set hbase.master.ui.readonly to false when kerberos is disabled. Use 
cluster-env to check if kerberos is enabled.


Bugs: AMBARI-16164
https://issues.apache.org/jira/browse/AMBARI-16164


Repository: ambari


Description
---

Currently in secure deployment, user can request compaction / splitting through 
hbase master UI.
This potentially exposes vulnerability to various attacks.
There is config parameter, hbase.master.ui.readonly, with default value of 
false.
In secure deployment, Master UI should be put to readonly mode (setting the 
above parameter to true).
Admin can always request compaction / splitting through hbase shell.


Diffs (updated)
-

  ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
689e1fd 

Diff: https://reviews.apache.org/r/46661/diff/


Testing
---

Manual testing, sceenshot attached (after kerberizing cluster).


File Attachments


hbase-site.png
  
https://reviews.apache.org/media/uploaded/files/2016/05/11/75e8d2ae-99a4-4e8d-8a69-52cd974734fb__hbase-site.png


Thanks,

Ajit Kumar

Review Request 47738: [Hive View] Non-visual explain gives a bunch of strangely clickable links

2016-05-23 Thread Pallav Kulshreshtha


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47738/
---

Review request for Ambari, DIPAYAN BHOWMICK and Nitiraj Rathore.


Bugs: AMBARI-16825
https://issues.apache.org/jira/browse/AMBARI-16825


Repository: ambari


Description
---

Removed unlickable links.


Diffs
-

  
contrib/views/hive/src/main/resources/ui/hive-web/app/controllers/index/history-query/explain.js
 20092e2 
  contrib/views/hive/src/main/resources/ui/hive-web/app/styles/app.scss be6d9ac 
  
contrib/views/hive/src/main/resources/ui/hive-web/app/templates/components/tree-view.hbs
 cb83ec3 
  
contrib/views/hive/src/main/resources/ui/hive-web/app/templates/index/history-query/explain.hbs
 3f8810f 

Diff: https://reviews.apache.org/r/47738/diff/


Testing
---

manually tested.


Thanks,

Pallav Kulshreshtha

Re: Review Request 47721: Editing local cluster name breaks all view instances

2016-05-23 Thread Pallav Kulshreshtha


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47721/#review134409
---


Ship it!




Ship It!

- Pallav Kulshreshtha


On May 23, 2016, 10:21 a.m., Gaurav Nagar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47721/
> ---
> 
> (Updated May 23, 2016, 10:21 a.m.)
> 
> 
> Review request for Ambari, DIPAYAN BHOWMICK, Nitiraj Rathore, Pallav 
> Kulshreshtha, Rohit Choudhary, and Ashwin Rajeev.
> 
> 
> Bugs: AMBARI-16812
> https://issues.apache.org/jira/browse/AMBARI-16812
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Changed cluster_handle in view instance to have cluster_id instead of 
> cluster_name.
> Made Remote cluster name editable
> 
> 
> Diffs
> -
> 
>   
> ambari-admin/src/main/resources/ui/admin-web/app/scripts/controllers/ambariViews/CreateViewInstanceCtrl.js
>  034b113 
>   
> ambari-admin/src/main/resources/ui/admin-web/app/scripts/controllers/ambariViews/ViewsEditCtrl.js
>  2b1fb3b 
>   
> ambari-admin/src/main/resources/ui/admin-web/app/scripts/controllers/remoteClusters/RemoteClustersEditCtrl.js
>  20eb552 
>   
> ambari-admin/src/main/resources/ui/admin-web/app/scripts/services/Cluster.js 
> 0bf73d5 
>   
> ambari-admin/src/main/resources/ui/admin-web/app/scripts/services/RemoteCluster.js
>  e4826e6 
>   ambari-admin/src/main/resources/ui/admin-web/app/scripts/services/View.js 
> dfe7ba8 
>   
> ambari-admin/src/main/resources/ui/admin-web/app/views/ambariViews/create.html
>  051d720 
>   
> ambari-admin/src/main/resources/ui/admin-web/app/views/ambariViews/edit.html 
> 3a69a50 
>   
> ambari-admin/src/main/resources/ui/admin-web/app/views/remoteClusters/editRemoteClusterPage.html
>  b5890bf 
>   
> ambari-admin/src/main/resources/ui/admin-web/test/unit/controllers/CreateViewInstanceCtrl.js
>  595fcc0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/ClusterNotFoundException.java
>  cd338a0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RemoteClusterResourceProvider.java
>  413dbff 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ViewInstanceResourceProvider.java
>  b8ed215 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/RemoteAmbariClusterDAO.java
>  72ab368 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RemoteAmbariClusterEntity.java
>  99c9f2a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/ViewInstanceEntity.java
>  98bafdd 
>   ambari-server/src/main/java/org/apache/ambari/server/state/Clusters.java 
> e660d20 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClustersImpl.java
>  f78dd95 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
>  41f538e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/view/RemoteAmbariClusterRegistry.java
>  38a47a4 
>   
> ambari-server/src/main/java/org/apache/ambari/server/view/ViewContextImpl.java
>  e98a4cd 
>   ambari-server/src/main/java/org/apache/ambari/server/view/ViewRegistry.java 
> 6d4ef82 
>   ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql 31dd8f2 
>   ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 3c4f1ca 
>   ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql f31af16 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql d1147b7 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql 
> 5a7a6a5 
>   ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 2dba3b7 
>   ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql e769e5b 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UserPrivilegeResourceProviderTest.java
>  88b00ca 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ViewInstanceResourceProviderTest.java
>  1044570 
>   
> ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AuthorizationHelperTest.java
>  9c59aab 
>   
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog240Test.java
>  20fa50f 
>   
> ambari-views/src/main/java/org/apache/ambari/view/ViewInstanceDefinition.java 
> cea4c88 
>   
> contrib/views/hive/src/main/java/org/apache/ambari/view/hive/PropertyValidator.java
>  ae73bc0 
>   
> contrib/views/pig/src/main/java/org/apache/ambari/view/pig/PropertyValidator.java
>  d3c9866 
> 
> Diff: https://reviews.apache.org/r/47721/diff/
> 
> 
> Testing
> ---
> 
> Manually tested
> 
> 
> Thanks,
> 
> Gaurav Nagar
> 
>

Re: Review Request 47565: AMBARI-16751: Refresh stale host entity reference when deleting a service

2016-05-23 Thread Alejandro Fernandez


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47565/#review134397
---


Ship it!




Ship It!

- Alejandro Fernandez


On May 23, 2016, 5:45 p.m., Nahappan Somasundaram wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47565/
> ---
> 
> (Updated May 23, 2016, 5:45 p.m.)
> 
> 
> Review request for Ambari, Myroslav Papirkovskyy, Sumit Mohanty, and Sid 
> Wagle.
> 
> 
> Bugs: AMBARI-16751
> https://issues.apache.org/jira/browse/AMBARI-16751
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> AMBARI-16751: Refresh stale host entity reference when deleting a service
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostComponentDesiredStateDAO.java
>  46da9dacf0e1c48249a83a167d5903f0ec100f7c 
>   
> ambari-server/src/test/java/org/apache/ambari/server/orm/dao/HostComponentDesiredStateDAOTest.java
>  b7a0677bcdd5ced0de9fd2bfcb7f290a6f6fe648 
> 
> Diff: https://reviews.apache.org/r/47565/diff/
> 
> 
> Testing
> ---
> 
> ** 1. mvn clean install **
> 
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Ambari Main ... SUCCESS [7.380s]
> [INFO] Apache Ambari Project POM . SUCCESS [0.036s]
> [INFO] Ambari Web  SUCCESS [23.544s]
> [INFO] Ambari Views .. SUCCESS [1.061s]
> [INFO] Ambari Admin View . SUCCESS [21.344s]
> [INFO] ambari-metrics  SUCCESS [0.602s]
> [INFO] Ambari Metrics Common . SUCCESS [0.912s]
> [INFO] Ambari Metrics Hadoop Sink  SUCCESS [2.140s]
> [INFO] Ambari Metrics Flume Sink . SUCCESS [1.109s]
> [INFO] Ambari Metrics Kafka Sink . SUCCESS [0.996s]
> [INFO] Ambari Metrics Storm Sink . SUCCESS [2.969s]
> [INFO] Ambari Metrics Collector .. SUCCESS [10.957s]
> [INFO] Ambari Metrics Monitor  SUCCESS [2.705s]
> [INFO] Ambari Metrics Grafana  SUCCESS [1.289s]
> [INFO] Ambari Metrics Assembly ... SUCCESS [1:31.486s]
> [INFO] Ambari Server . SUCCESS [2:27.145s]
> [INFO] Ambari Functional Tests ... SUCCESS [1.128s]
> [INFO] Ambari Agent .. SUCCESS [23.899s]
> [INFO] Ambari Client . SUCCESS [0.047s]
> [INFO] Ambari Python Client .. SUCCESS [0.993s]
> [INFO] Ambari Groovy Client .. SUCCESS [2.117s]
> [INFO] Ambari Shell .. SUCCESS [0.078s]
> [INFO] Ambari Python Shell ... SUCCESS [0.717s]
> [INFO] Ambari Groovy Shell ... SUCCESS [0.841s]
> [INFO] ambari-logsearch .. SUCCESS [0.191s]
> [INFO] Ambari Logsearch Appender . SUCCESS [0.223s]
> [INFO] Ambari Logsearch Solr Client .. SUCCESS [1.043s]
> [INFO] Ambari Logsearch Portal ... SUCCESS [5.856s]
> [INFO] Ambari Logsearch Log Feeder ... SUCCESS [2.262s]
> [INFO] Ambari Logsearch Assembly . SUCCESS [0.119s]
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 5:55.888s
> [INFO] Finished at: Wed May 18 16:42:38 PDT 2016
> [INFO] Final Memory: 290M/1262M
> [INFO] 
> 
> 
> ** 2. mvn test -Dtest=HostComponentDesiredStateDAOTest -DskipPythonTests **
> 
> ---
>  T E S T S
> ---
> Picked up _JAVA_OPTIONS: -Xmx2048m -XX:MaxPermSize=512m 
> -Djava.awt.headless=true
> Running org.apache.ambari.server.orm.dao.HostComponentDesiredStateDAOTest
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.209 sec - 
> in org.apache.ambari.server.orm.dao.HostComponentDesiredStateDAOTest
> 
> Results :
> 
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
>

Re: Review Request 47565: AMBARI-16751: Refresh stale host entity reference when deleting a service

2016-05-23 Thread Nahappan Somasundaram


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47565/
---

(Updated May 23, 2016, 10:45 a.m.)


Review request for Ambari, Myroslav Papirkovskyy, Sumit Mohanty, and Sid Wagle.


Changes
---

If host entity is null, throw IllegalStateException.


Bugs: AMBARI-16751
https://issues.apache.org/jira/browse/AMBARI-16751


Repository: ambari


Description
---

AMBARI-16751: Refresh stale host entity reference when deleting a service


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostComponentDesiredStateDAO.java
 46da9dacf0e1c48249a83a167d5903f0ec100f7c 
  
ambari-server/src/test/java/org/apache/ambari/server/orm/dao/HostComponentDesiredStateDAOTest.java
 b7a0677bcdd5ced0de9fd2bfcb7f290a6f6fe648 

Diff: https://reviews.apache.org/r/47565/diff/


Testing
---

** 1. mvn clean install **

[INFO] 
[INFO] Reactor Summary:
[INFO]
[INFO] Ambari Main ... SUCCESS [7.380s]
[INFO] Apache Ambari Project POM . SUCCESS [0.036s]
[INFO] Ambari Web  SUCCESS [23.544s]
[INFO] Ambari Views .. SUCCESS [1.061s]
[INFO] Ambari Admin View . SUCCESS [21.344s]
[INFO] ambari-metrics  SUCCESS [0.602s]
[INFO] Ambari Metrics Common . SUCCESS [0.912s]
[INFO] Ambari Metrics Hadoop Sink  SUCCESS [2.140s]
[INFO] Ambari Metrics Flume Sink . SUCCESS [1.109s]
[INFO] Ambari Metrics Kafka Sink . SUCCESS [0.996s]
[INFO] Ambari Metrics Storm Sink . SUCCESS [2.969s]
[INFO] Ambari Metrics Collector .. SUCCESS [10.957s]
[INFO] Ambari Metrics Monitor  SUCCESS [2.705s]
[INFO] Ambari Metrics Grafana  SUCCESS [1.289s]
[INFO] Ambari Metrics Assembly ... SUCCESS [1:31.486s]
[INFO] Ambari Server . SUCCESS [2:27.145s]
[INFO] Ambari Functional Tests ... SUCCESS [1.128s]
[INFO] Ambari Agent .. SUCCESS [23.899s]
[INFO] Ambari Client . SUCCESS [0.047s]
[INFO] Ambari Python Client .. SUCCESS [0.993s]
[INFO] Ambari Groovy Client .. SUCCESS [2.117s]
[INFO] Ambari Shell .. SUCCESS [0.078s]
[INFO] Ambari Python Shell ... SUCCESS [0.717s]
[INFO] Ambari Groovy Shell ... SUCCESS [0.841s]
[INFO] ambari-logsearch .. SUCCESS [0.191s]
[INFO] Ambari Logsearch Appender . SUCCESS [0.223s]
[INFO] Ambari Logsearch Solr Client .. SUCCESS [1.043s]
[INFO] Ambari Logsearch Portal ... SUCCESS [5.856s]
[INFO] Ambari Logsearch Log Feeder ... SUCCESS [2.262s]
[INFO] Ambari Logsearch Assembly . SUCCESS [0.119s]
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 5:55.888s
[INFO] Finished at: Wed May 18 16:42:38 PDT 2016
[INFO] Final Memory: 290M/1262M
[INFO] 

** 2. mvn test -Dtest=HostComponentDesiredStateDAOTest -DskipPythonTests **

---
 T E S T S
---
Picked up _JAVA_OPTIONS: -Xmx2048m -XX:MaxPermSize=512m -Djava.awt.headless=true
Running org.apache.ambari.server.orm.dao.HostComponentDesiredStateDAOTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.209 sec - in 
org.apache.ambari.server.orm.dao.HostComponentDesiredStateDAOTest

Results :

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 24.701s
[INFO] Finished at: Thu May 19 14:59:41 PDT 2016
[INFO] Final Memory: 57M/1024M
[INFO] 


Thanks,

Nahappan Somasundaram

Re: Review Request 47734: AMBARI-15898 : Grafana fails to start when using custom data directory

2016-05-23 Thread Dmytro Sen


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47734/#review134389
---


Ship it!




Ship It!

- Dmytro Sen


On Май 23, 2016, 5:10 п.п., Aravindan Vijayan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47734/
> ---
> 
> (Updated Май 23, 2016, 5:10 п.п.)
> 
> 
> Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-15898
> https://issues.apache.org/jira/browse/AMBARI-15898
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> STR:
> 1. Deploy using blueprint with custom data directory for grafana. 
> {code}
> {
>   "ams-grafana-env": {
> "properties": {
>   "metrics_grafana_username": "admin",
>   "metrics_grafana_password": "admin",
>   "metrics_grafana_log_dir": "/var/opt/log/ambari-metrics-grafana",
>   "metrics_grafana_data_dir": "/var/opt/lib/ambari-metrics-grafana"
> }
>   }
> },{code}
> 2. Grafana start fails.
> {code}
> 2016/04/14 17:31:52 [I] Starting Grafana
> 2016/04/14 17:31:52 [I] Version: 2.6.0, Commit: v2.6.0, Build date: 
> 2015-12-14 09:18:01 -0500 EST
> 2016/04/14 17:31:52 [I] Configuration Info
> Config files:
>   [0]: /usr/lib/ambari-metrics-grafana/conf/defaults.ini
>   [1]: /etc/ambari-metrics-grafana/conf/ams-grafana.ini
> Command lines overrides:
>   [0]: default.paths.data=/var/opt/lib/ambari-metrics-grafana
>   [1]: default.paths.logs=/var/opt/log/ambari-metrics-grafana
> Paths:
>   home: /usr/lib/ambari-metrics-grafana
>   data: /var/lib/ambari-metrics-grafana
>   logs: /var/opt/log/ambari-metrics-grafana
> 
> 2016/04/14 17:31:52 [I] Database: sqlite3
> 2016/04/14 17:31:52 [I] Migrator: Starting DB migration
> 2016/04/14 17:31:52 [log.go:75 Fatal()] [E] fail to initialize orm engine: 
> Sqlstore::Migration failed err: unable to open database file
> {code}
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
>  57cfa26 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-grafana-ini.xml
>  9e05860 
>   
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog240Test.java
>  606fc72 
> 
> Diff: https://reviews.apache.org/r/47734/diff/
> 
> 
> Testing
> ---
> 
> Manually tested.
> 
> 
> Thanks,
> 
> Aravindan Vijayan
> 
>

Review Request 47734: AMBARI-15898 : Grafana fails to start when using custom data directory

2016-05-23 Thread Aravindan Vijayan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47734/
---

Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle.


Bugs: AMBARI-15898
https://issues.apache.org/jira/browse/AMBARI-15898


Repository: ambari


Description
---

STR:
1. Deploy using blueprint with custom data directory for grafana. 
{code}
{
  "ams-grafana-env": {
"properties": {
  "metrics_grafana_username": "admin",
  "metrics_grafana_password": "admin",
  "metrics_grafana_log_dir": "/var/opt/log/ambari-metrics-grafana",
  "metrics_grafana_data_dir": "/var/opt/lib/ambari-metrics-grafana"
}
  }
},{code}
2. Grafana start fails.
{code}
2016/04/14 17:31:52 [I] Starting Grafana
2016/04/14 17:31:52 [I] Version: 2.6.0, Commit: v2.6.0, Build date: 2015-12-14 
09:18:01 -0500 EST
2016/04/14 17:31:52 [I] Configuration Info
Config files:
  [0]: /usr/lib/ambari-metrics-grafana/conf/defaults.ini
  [1]: /etc/ambari-metrics-grafana/conf/ams-grafana.ini
Command lines overrides:
  [0]: default.paths.data=/var/opt/lib/ambari-metrics-grafana
  [1]: default.paths.logs=/var/opt/log/ambari-metrics-grafana
Paths:
  home: /usr/lib/ambari-metrics-grafana
  data: /var/lib/ambari-metrics-grafana
  logs: /var/opt/log/ambari-metrics-grafana

2016/04/14 17:31:52 [I] Database: sqlite3
2016/04/14 17:31:52 [I] Migrator: Starting DB migration
2016/04/14 17:31:52 [log.go:75 Fatal()] [E] fail to initialize orm engine: 
Sqlstore::Migration failed err: unable to open database file
{code}


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
 57cfa26 
  
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-grafana-ini.xml
 9e05860 
  
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog240Test.java
 606fc72 

Diff: https://reviews.apache.org/r/47734/diff/


Testing
---

Manually tested.


Thanks,

Aravindan Vijayan

Re: Review Request 45507: Enhance blueprint support for using references

2016-05-23 Thread Robert Nettleton

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45507/#review134360
---

I have some general concerns about this patch, and the proposed overall 
approach, and I've listed out my concerns on the following JIRA:

https://issues.apache.org/jira/browse/AMBARI-15395

The issues listed below are related to the concerns I've posted in the link 
above. 

Thanks.

File Attachment: AMBARI-15406-v1 - AMBARI-15406.patch

I'm not sure this makes sense. 

The method here should never rely on specify implementations of the Filter 
interface.  

In general, I think that passwords should still be filtered out, since the 
Blueprint/Cluster Creation template validation code will run 
error-checks/validation checks, and notify the user/blueprint deployer that a 
password is missing.

File Attachment: AMBARI-15406-v1 - AMBARI-15406.patch

I'm still not sure I agree with this approach generally. 

The "Secret Reference" format used by the Ambari REST API doesn't really 
translate well to be included within a Blueprint.  The reference syntax 
includes things like configuration type, property name, etc, which are 
generally already included in the Blueprint.

As with my comment above, it's my opinion that passwords should be 
excluded.  The Blueprints processor already has checks to fail a deployment 
attempt if a password is missing, and already has the support for the "default 
password" feature in non-production environments.

- Robert Nettleton

On May 12, 2016, 9:40 p.m., Amruta Borkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45507/
> ---
> 
> (Updated May 12, 2016, 9:40 p.m.)
> 
> 
> Review request for Ambari, Jayush Luniya, Robert Nettleton, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-15395 and AMBARI-15406
> https://issues.apache.org/jira/browse/AMBARI-15395
> https://issues.apache.org/jira/browse/AMBARI-15406
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Enhance blueprints to export placeholder/token for password properties. This 
> is to avoid user from having tochange the password once the cluster is formed 
> if a user wishes to do so.
> Secret References acting as tokens to password properties would be replaced 
> by user with appropriate passwords,
> If any Secret references are found during cluster deployment with blueprint, 
> those will be replaced by default_password provided in blueprint template. 
> Need more comments to make this feature more portable.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
>  9cc7b13 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintValidatorImpl.java
>  432c6f8 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/RequiredPasswordValidator.java
>  98eaa40 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
>  14a718d 
>   
> ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintImplTest.java
>  0b06eb8 
>   
> ambari-server/src/test/java/org/apache/ambari/server/topology/RequiredPasswordValidatorTest.java
>  e8a2ff5 
> 
> Diff: https://reviews.apache.org/r/45507/diff/
> 
> 
> Testing
> ---
> 
> Tested the patch by trying out blueprint export and creating a cluster from 
> the exported blueprint. Result was: the password tokens were replaced by 
> default password and cluster was created successfully.
> 
> 
> File Attachments
> 
> 
> AMBARI-15406.patch
>   
> https://reviews.apache.org/media/uploaded/files/2016/03/30/8daa09ee-82e5-4cd8-98d3-b1c00f1269b7__AMBARI-15406.patch
> AMBARI-15406-v1
>   
> https://reviews.apache.org/media/uploaded/files/2016/05/12/7faf76cc-a42d-4017-b203-8db19448b09d__AMBARI-15406.patch
> 
> 
> Thanks,
> 
> Amruta Borkar
> 
>

Re: Review Request 47701: AMBARI-16642: After enabling HTTPS for YARN, NodeManager JVM Metrics on YARN Heatmaps show NA.

2016-05-23 Thread Qin Liu



> On May 23, 2016, 3:49 p.m., Sid Wagle wrote:
> > Ship It!
> 
> Sid Wagle wrote:
> Pushed to trunk on behalf on Qin Liu, REview can be closed. Thanks.

Sid, thank you so much for your help on this! I really appreciated it.


- Qin


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47701/#review134366
---


On May 23, 2016, 5:59 a.m., Qin Liu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47701/
> ---
> 
> (Updated May 23, 2016, 5:59 a.m.)
> 
> 
> Review request for Ambari, Dmytro Sen and Sid Wagle.
> 
> 
> Bugs: AMBARI-16642
> https://issues.apache.org/jira/browse/AMBARI-16642
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> After enabling HTTPS for YARN, "NodeManager GC Time" and "NodeManager JVM 
> Heap Memory Used" widgets on YARN Heatmaps show NA. The root cause is that 
> yarn.nodemanager.webapp.address rather than 
> yarn.nodemanager.webapp.https.address is used to get JMX metrics. 
> 
> Steps to reproduce:
> 1. install a cluster with default from Ambari Web UI.
> 2. configuring SSL for HDFS, YARN, and MapReduce.
> 3. enable HTTPS for YARN
> 1). Set the following property in Advanced yarn-site from Ambari Web UI:
> yarn.http.policy=HTTPS_ONLY
> 2). Add the following property in Custom yarn-site from Ambari Web UI:
> yarn.nodemanager.webapp.https.address=0.0.0.0:8044
> 4. "NodeManager GC Time" and "NodeManager JVM Heap Memory Used" widgets on 
> YARN Heatmaps will show NA.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractProviderModule.java
>  36be39e 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/JMXHostProviderTest.java
>  a6e5501 
> 
> Diff: https://reviews.apache.org/r/47701/diff/
> 
> 
> Testing
> ---
> 
> 1. The fix has been tested with HDP UI
> "NodeManager GC Time" and "NodeManager JVM Heap Memory Used" metrics show up 
> after apply the fix.
> 
> 2. JMXHostProviderTest.java has been modified to verify JMX protocol and port 
> for NodeManager.
> Test set: org.apache.ambari.server.controller.internal.JMXHostProviderTest
> ---
> Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 120.274 sec - 
> in org.apache.ambari.server.controller.internal.JMXHostProviderTest
> 
> 3. The latest result of running ambari-server tests:
> Results :
> Failed tests: 
>   DataStoreImplTest.testFind:409 
>   Unexpected method call EntityManager.getDelegate():
> EntityManager.find(capture(Nothing captured yet), 99): expected: 1, 
> actual: 0
> EntityManager.close(): expected: 1, actual: 0
>   DataStoreImplTest.testFindAll:453 
>   Unexpected method call EntityManager.getDelegate():
> EntityManager.createQuery("SELECT e FROM 
> DS_DataStoreImplTest$TestEntity_1 e WHERE e.DS_id=99"): expected: 1, actual: 0
> EntityManager.close(): expected: 1, actual: 0
>   DataStoreImplTest.testFindAll_multiple:518 
>   Unexpected method call EntityManager.getDelegate():
> EntityManager.createQuery("SELECT e FROM 
> DS_DataStoreImplTest$TestEntity_1 e WHERE e.DS_name='foo'"): expected: 1, 
> actual: 0
> EntityManager.close(): expected: 1, actual: 0
>   DataStoreImplTest.testRemove:370 
>   Unexpected method call EntityManager.getDelegate():
> EntityManager.getReference(capture(Nothing captured yet), 99): expected: 
> 1, actual: 0
> EntityManager.remove(EasyMock for interface 
> org.eclipse.persistence.dynamic.DynamicEntity): expected: 1, actual: 0
> EntityManager.close(): expected: 1, actual: 0
>   DataStoreImplTest.testStore_create:128 
>   Unexpected method call EntityManager.getDelegate():
> EntityManager.find(capture(Nothing captured yet), "bar"): expected: 1, 
> actual: 0
> EntityManager.find(capture(Nothing captured yet), 99): expected: 1, 
> actual: 0
> EntityManager.persist(capture(Nothing captured yet)): expected: 1, 
> actual: 0
> EntityManager.persist(capture(Nothing captured yet)): expected: 1, 
> actual: 0
> EntityManager.close(): expected: 1, actual: 0
>   DataStoreImplTest.testStore_create_largeEntity:222 
>   Unexpected method call EntityManager.getDelegate():
> EntityManager.close(): expected: 1, actual: 0
>   DataStoreImplTest.testStore_create_longStringValue:182 
>   Unexpected method call EntityManager.getDelegate():
> EntityManager.find(capture(Nothing captured yet), 99): expected: 1, 
> actual: 0
> EntityManager.close(): expected: 1, actual: 0
>   DataStoreImplTest.testStore_update:276 
>   Unexpected method call EntityManager.getDelegate():
> EntityManager.find(capture(Nothing captured yet), "bar"):

Re: Review Request 47565: AMBARI-16751: Refresh stale host entity reference when deleting a service

2016-05-23 Thread Sid Wagle


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47565/#review134378
---




ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostComponentDesiredStateDAO.java
 (line 98)


Current delete impl do not remove host before hostcomponent but it might be 
appropriate to throw a UnChecked exception here vs a NPE if host is already 
removed. Optional for this patch since such code path is non-existent.


- Sid Wagle


On May 19, 2016, 10:04 p.m., Nahappan Somasundaram wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47565/
> ---
> 
> (Updated May 19, 2016, 10:04 p.m.)
> 
> 
> Review request for Ambari, Myroslav Papirkovskyy, Sumit Mohanty, and Sid 
> Wagle.
> 
> 
> Bugs: AMBARI-16751
> https://issues.apache.org/jira/browse/AMBARI-16751
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> AMBARI-16751: Refresh stale host entity reference when deleting a service
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostComponentDesiredStateDAO.java
>  46da9dacf0e1c48249a83a167d5903f0ec100f7c 
>   
> ambari-server/src/test/java/org/apache/ambari/server/orm/dao/HostComponentDesiredStateDAOTest.java
>  b7a0677bcdd5ced0de9fd2bfcb7f290a6f6fe648 
> 
> Diff: https://reviews.apache.org/r/47565/diff/
> 
> 
> Testing
> ---
> 
> ** 1. mvn clean install **
> 
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Ambari Main ... SUCCESS [7.380s]
> [INFO] Apache Ambari Project POM . SUCCESS [0.036s]
> [INFO] Ambari Web  SUCCESS [23.544s]
> [INFO] Ambari Views .. SUCCESS [1.061s]
> [INFO] Ambari Admin View . SUCCESS [21.344s]
> [INFO] ambari-metrics  SUCCESS [0.602s]
> [INFO] Ambari Metrics Common . SUCCESS [0.912s]
> [INFO] Ambari Metrics Hadoop Sink  SUCCESS [2.140s]
> [INFO] Ambari Metrics Flume Sink . SUCCESS [1.109s]
> [INFO] Ambari Metrics Kafka Sink . SUCCESS [0.996s]
> [INFO] Ambari Metrics Storm Sink . SUCCESS [2.969s]
> [INFO] Ambari Metrics Collector .. SUCCESS [10.957s]
> [INFO] Ambari Metrics Monitor  SUCCESS [2.705s]
> [INFO] Ambari Metrics Grafana  SUCCESS [1.289s]
> [INFO] Ambari Metrics Assembly ... SUCCESS [1:31.486s]
> [INFO] Ambari Server . SUCCESS [2:27.145s]
> [INFO] Ambari Functional Tests ... SUCCESS [1.128s]
> [INFO] Ambari Agent .. SUCCESS [23.899s]
> [INFO] Ambari Client . SUCCESS [0.047s]
> [INFO] Ambari Python Client .. SUCCESS [0.993s]
> [INFO] Ambari Groovy Client .. SUCCESS [2.117s]
> [INFO] Ambari Shell .. SUCCESS [0.078s]
> [INFO] Ambari Python Shell ... SUCCESS [0.717s]
> [INFO] Ambari Groovy Shell ... SUCCESS [0.841s]
> [INFO] ambari-logsearch .. SUCCESS [0.191s]
> [INFO] Ambari Logsearch Appender . SUCCESS [0.223s]
> [INFO] Ambari Logsearch Solr Client .. SUCCESS [1.043s]
> [INFO] Ambari Logsearch Portal ... SUCCESS [5.856s]
> [INFO] Ambari Logsearch Log Feeder ... SUCCESS [2.262s]
> [INFO] Ambari Logsearch Assembly . SUCCESS [0.119s]
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 5:55.888s
> [INFO] Finished at: Wed May 18 16:42:38 PDT 2016
> [INFO] Final Memory: 290M/1262M
> [INFO] 
> 
> 
> ** 2. mvn test -Dtest=HostComponentDesiredStateDAOTest -DskipPythonTests **
> 
> ---
>  T E S T S
> ---
> Picked up _JAVA_OPTIONS: -Xmx2048m -XX:MaxPermSize=512m 
> -Djava.awt.headless=true
> Running org.apache.ambari.server.orm.dao.HostComponentDesiredStateDAOTest
> Tests run: 1, Failures: 0, Errors: 0, Skipped:

Re: Review Request 47565: AMBARI-16751: Refresh stale host entity reference when deleting a service

2016-05-23 Thread Sid Wagle


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47565/#review134377
---


Ship it!




Ship It!

- Sid Wagle


On May 19, 2016, 10:04 p.m., Nahappan Somasundaram wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47565/
> ---
> 
> (Updated May 19, 2016, 10:04 p.m.)
> 
> 
> Review request for Ambari, Myroslav Papirkovskyy, Sumit Mohanty, and Sid 
> Wagle.
> 
> 
> Bugs: AMBARI-16751
> https://issues.apache.org/jira/browse/AMBARI-16751
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> AMBARI-16751: Refresh stale host entity reference when deleting a service
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostComponentDesiredStateDAO.java
>  46da9dacf0e1c48249a83a167d5903f0ec100f7c 
>   
> ambari-server/src/test/java/org/apache/ambari/server/orm/dao/HostComponentDesiredStateDAOTest.java
>  b7a0677bcdd5ced0de9fd2bfcb7f290a6f6fe648 
> 
> Diff: https://reviews.apache.org/r/47565/diff/
> 
> 
> Testing
> ---
> 
> ** 1. mvn clean install **
> 
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Ambari Main ... SUCCESS [7.380s]
> [INFO] Apache Ambari Project POM . SUCCESS [0.036s]
> [INFO] Ambari Web  SUCCESS [23.544s]
> [INFO] Ambari Views .. SUCCESS [1.061s]
> [INFO] Ambari Admin View . SUCCESS [21.344s]
> [INFO] ambari-metrics  SUCCESS [0.602s]
> [INFO] Ambari Metrics Common . SUCCESS [0.912s]
> [INFO] Ambari Metrics Hadoop Sink  SUCCESS [2.140s]
> [INFO] Ambari Metrics Flume Sink . SUCCESS [1.109s]
> [INFO] Ambari Metrics Kafka Sink . SUCCESS [0.996s]
> [INFO] Ambari Metrics Storm Sink . SUCCESS [2.969s]
> [INFO] Ambari Metrics Collector .. SUCCESS [10.957s]
> [INFO] Ambari Metrics Monitor  SUCCESS [2.705s]
> [INFO] Ambari Metrics Grafana  SUCCESS [1.289s]
> [INFO] Ambari Metrics Assembly ... SUCCESS [1:31.486s]
> [INFO] Ambari Server . SUCCESS [2:27.145s]
> [INFO] Ambari Functional Tests ... SUCCESS [1.128s]
> [INFO] Ambari Agent .. SUCCESS [23.899s]
> [INFO] Ambari Client . SUCCESS [0.047s]
> [INFO] Ambari Python Client .. SUCCESS [0.993s]
> [INFO] Ambari Groovy Client .. SUCCESS [2.117s]
> [INFO] Ambari Shell .. SUCCESS [0.078s]
> [INFO] Ambari Python Shell ... SUCCESS [0.717s]
> [INFO] Ambari Groovy Shell ... SUCCESS [0.841s]
> [INFO] ambari-logsearch .. SUCCESS [0.191s]
> [INFO] Ambari Logsearch Appender . SUCCESS [0.223s]
> [INFO] Ambari Logsearch Solr Client .. SUCCESS [1.043s]
> [INFO] Ambari Logsearch Portal ... SUCCESS [5.856s]
> [INFO] Ambari Logsearch Log Feeder ... SUCCESS [2.262s]
> [INFO] Ambari Logsearch Assembly . SUCCESS [0.119s]
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 5:55.888s
> [INFO] Finished at: Wed May 18 16:42:38 PDT 2016
> [INFO] Final Memory: 290M/1262M
> [INFO] 
> 
> 
> ** 2. mvn test -Dtest=HostComponentDesiredStateDAOTest -DskipPythonTests **
> 
> ---
>  T E S T S
> ---
> Picked up _JAVA_OPTIONS: -Xmx2048m -XX:MaxPermSize=512m 
> -Djava.awt.headless=true
> Running org.apache.ambari.server.orm.dao.HostComponentDesiredStateDAOTest
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.209 sec - 
> in org.apache.ambari.server.orm.dao.HostComponentDesiredStateDAOTest
> 
> Results :
> 
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
>

Re: Review Request 47726: Blueprint Export does not replace hive.llap.zk.sm.connectionString

2016-05-23 Thread Robert Nettleton



> On May 23, 2016, 2:59 p.m., Robert Nettleton wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java,
> >  line 2395
> > 
> >
> > +1 for Sebastian's comment, a unit test should definitely be added here 
> > to verify this change.
> 
> Daniel Gergely wrote:
> We were talking about this unit test thing. These are static variables, 
> so I can test if all the properties are there in the map. However it is not 
> easy to test whether the correct PropertyUpdater is used with the correct 
> component. These are not visible from outside, so a complex mocking or using 
> Reflection could made it possible, but these are messy things. So he dropped 
> the issue here.
> 
> Robert Nettleton wrote:
> Hi Daniel, 
> 
> It should be pretty simple to unit test this issue, by verifiying that 
> this property is updated as expected, when the expected set of components are 
> included in the Blueprint/Cluster Configuration.  There is no need to verify 
> the static maps being modified, but there is definitely a need to make sure 
> that this property is handled correctly by the Blueprint export process, so I 
> still think a unit test is necessary here. 
> 
> Thanks.

Here's an example of a unit test that verifies that the property updaters are 
handling Blueprint exports correctly:

org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessorTest#testYarnConfigExported

There are many more examples of this in this test case as well.

Thanks.


- Robert


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47726/#review134358
---


On May 23, 2016, 2:07 p.m., Daniel Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47726/
> ---
> 
> (Updated May 23, 2016, 2:07 p.m.)
> 
> 
> Review request for Ambari, Balázs Bence Sári, Laszlo Puskas, Oliver Szabo, 
> Robert Nettleton, Sandor Magyari, Sumit Mohanty, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-16820
> https://issues.apache.org/jira/browse/AMBARI-16820
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> MultiHostUpdater was not assigned to the property 
> hive.llap.zk.sm.connectionString
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
>  9cabdbb 
> 
> Diff: https://reviews.apache.org/r/47726/diff/
> 
> 
> Testing
> ---
> 
> In progress...
> 
> 
> Thanks,
> 
> Daniel Gergely
> 
>

Re: Review Request 47701: AMBARI-16642: After enabling HTTPS for YARN, NodeManager JVM Metrics on YARN Heatmaps show NA.

2016-05-23 Thread Sid Wagle



> On May 23, 2016, 3:49 p.m., Sid Wagle wrote:
> > Ship It!

Pushed to trunk on behalf on Qin Liu, REview can be closed. Thanks.


- Sid


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47701/#review134366
---


On May 23, 2016, 5:59 a.m., Qin Liu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47701/
> ---
> 
> (Updated May 23, 2016, 5:59 a.m.)
> 
> 
> Review request for Ambari, Dmytro Sen and Sid Wagle.
> 
> 
> Bugs: AMBARI-16642
> https://issues.apache.org/jira/browse/AMBARI-16642
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> After enabling HTTPS for YARN, "NodeManager GC Time" and "NodeManager JVM 
> Heap Memory Used" widgets on YARN Heatmaps show NA. The root cause is that 
> yarn.nodemanager.webapp.address rather than 
> yarn.nodemanager.webapp.https.address is used to get JMX metrics. 
> 
> Steps to reproduce:
> 1. install a cluster with default from Ambari Web UI.
> 2. configuring SSL for HDFS, YARN, and MapReduce.
> 3. enable HTTPS for YARN
> 1). Set the following property in Advanced yarn-site from Ambari Web UI:
> yarn.http.policy=HTTPS_ONLY
> 2). Add the following property in Custom yarn-site from Ambari Web UI:
> yarn.nodemanager.webapp.https.address=0.0.0.0:8044
> 4. "NodeManager GC Time" and "NodeManager JVM Heap Memory Used" widgets on 
> YARN Heatmaps will show NA.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractProviderModule.java
>  36be39e 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/JMXHostProviderTest.java
>  a6e5501 
> 
> Diff: https://reviews.apache.org/r/47701/diff/
> 
> 
> Testing
> ---
> 
> 1. The fix has been tested with HDP UI
> "NodeManager GC Time" and "NodeManager JVM Heap Memory Used" metrics show up 
> after apply the fix.
> 
> 2. JMXHostProviderTest.java has been modified to verify JMX protocol and port 
> for NodeManager.
> Test set: org.apache.ambari.server.controller.internal.JMXHostProviderTest
> ---
> Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 120.274 sec - 
> in org.apache.ambari.server.controller.internal.JMXHostProviderTest
> 
> 3. The latest result of running ambari-server tests:
> Results :
> Failed tests: 
>   DataStoreImplTest.testFind:409 
>   Unexpected method call EntityManager.getDelegate():
> EntityManager.find(capture(Nothing captured yet), 99): expected: 1, 
> actual: 0
> EntityManager.close(): expected: 1, actual: 0
>   DataStoreImplTest.testFindAll:453 
>   Unexpected method call EntityManager.getDelegate():
> EntityManager.createQuery("SELECT e FROM 
> DS_DataStoreImplTest$TestEntity_1 e WHERE e.DS_id=99"): expected: 1, actual: 0
> EntityManager.close(): expected: 1, actual: 0
>   DataStoreImplTest.testFindAll_multiple:518 
>   Unexpected method call EntityManager.getDelegate():
> EntityManager.createQuery("SELECT e FROM 
> DS_DataStoreImplTest$TestEntity_1 e WHERE e.DS_name='foo'"): expected: 1, 
> actual: 0
> EntityManager.close(): expected: 1, actual: 0
>   DataStoreImplTest.testRemove:370 
>   Unexpected method call EntityManager.getDelegate():
> EntityManager.getReference(capture(Nothing captured yet), 99): expected: 
> 1, actual: 0
> EntityManager.remove(EasyMock for interface 
> org.eclipse.persistence.dynamic.DynamicEntity): expected: 1, actual: 0
> EntityManager.close(): expected: 1, actual: 0
>   DataStoreImplTest.testStore_create:128 
>   Unexpected method call EntityManager.getDelegate():
> EntityManager.find(capture(Nothing captured yet), "bar"): expected: 1, 
> actual: 0
> EntityManager.find(capture(Nothing captured yet), 99): expected: 1, 
> actual: 0
> EntityManager.persist(capture(Nothing captured yet)): expected: 1, 
> actual: 0
> EntityManager.persist(capture(Nothing captured yet)): expected: 1, 
> actual: 0
> EntityManager.close(): expected: 1, actual: 0
>   DataStoreImplTest.testStore_create_largeEntity:222 
>   Unexpected method call EntityManager.getDelegate():
> EntityManager.close(): expected: 1, actual: 0
>   DataStoreImplTest.testStore_create_longStringValue:182 
>   Unexpected method call EntityManager.getDelegate():
> EntityManager.find(capture(Nothing captured yet), 99): expected: 1, 
> actual: 0
> EntityManager.close(): expected: 1, actual: 0
>   DataStoreImplTest.testStore_update:276 
>   Unexpected method call EntityManager.getDelegate():
> EntityManager.find(capture(Nothing captured yet), "bar"): expected: 1, 
> actual: 0
> EntityManager.find(capture(Nothing captured yet), 99): expected: 1, 
>

Re: Review Request 47730: Improve TimelineMetricsCache eviction/flush logic using a cache library

2016-05-23 Thread Sid Wagle


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47730/#review134370
---




ambari-metrics/ambari-metrics-common/pom.xml (line 78)


This will collide due to Hadoop dependency in version 14.0.1, I faced a 
problem here before. Can we make this shaded ?


- Sid Wagle


On May 23, 2016, 3:48 p.m., Dmytro Sen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47730/
> ---
> 
> (Updated May 23, 2016, 3:48 p.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Myroslav Papirkovskyy, and Sid 
> Wagle.
> 
> 
> Bugs: AMBARI-16821
> https://issues.apache.org/jira/browse/AMBARI-16821
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> The TimelineMetricsCache implementation in the metrics sink side is currently 
> a ConcurrentSkipListMap. It is better to use pre built cache libraries like 
> Guava that offer more support.
> 
> 
> Diffs
> -
> 
>   ambari-metrics/ambari-metrics-common/pom.xml 5a388dd 
>   
> ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/cache/TimelineMetricsCache.java
>  0bed7d0 
>   
> ambari-metrics/ambari-metrics-common/src/test/java/org/apache/hadoop/metrics2/sink/timeline/cache/TimelineMetricsCacheTest.java
>  18d973c 
> 
> Diff: https://reviews.apache.org/r/47730/diff/
> 
> 
> Testing
> ---
> 
> Unit tests passed
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Review Request 47730: Improve TimelineMetricsCache eviction/flush logic using a cache library

2016-05-23 Thread Dmytro Sen


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47730/
---

Review request for Ambari, Aravindan Vijayan, Myroslav Papirkovskyy, and Sid 
Wagle.


Bugs: AMBARI-16821
https://issues.apache.org/jira/browse/AMBARI-16821


Repository: ambari


Description
---

The TimelineMetricsCache implementation in the metrics sink side is currently a 
ConcurrentSkipListMap. It is better to use pre built cache libraries like Guava 
that offer more support.


Diffs
-

  ambari-metrics/ambari-metrics-common/pom.xml 5a388dd 
  
ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/cache/TimelineMetricsCache.java
 0bed7d0 
  
ambari-metrics/ambari-metrics-common/src/test/java/org/apache/hadoop/metrics2/sink/timeline/cache/TimelineMetricsCacheTest.java
 18d973c 

Diff: https://reviews.apache.org/r/47730/diff/


Testing
---

Unit tests passed


Thanks,

Dmytro Sen

Re: Review Request 47729: MAPREDUCE2_SERVICE_CHECK failed on secured cluster

2016-05-23 Thread Nate Cole


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47729/#review134363
---


Ship it!




Ship It!

- Nate Cole


On May 23, 2016, 11:32 a.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47729/
> ---
> 
> (Updated May 23, 2016, 11:32 a.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Nate Cole, and Sandor Magyari.
> 
> 
> Bugs: AMBARI-16801
> https://issues.apache.org/jira/browse/AMBARI-16801
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> # Error
> ```
> "stderr" : "Traceback (most recent call last):\n  File 
> \"/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/mapred_service_check.py\",
>  line 162, in \nMapReduce2ServiceCheck().execute()\n  File 
> \"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py\",
>  line 254, in execute\nmethod(env)\n  File 
> \"/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/mapred_service_check.py\",
>  line 140, in service_check\nparams.smokeuser_principal, params.hostname, 
> params.tmp_dir)\n  File 
> \"/usr/lib/python2.6/site-packages/resource_management/libraries/functions/security_commons.py\",
>  line 204, in cached_kinit_executor\nnew_cached_exec(key, file_path, 
> kinit_path, temp_dir, exec_user, keytab_file, principal, hostname)\n  File 
> \"/usr/lib/python2.6/site-packages/resource_management/libraries/functions/security_commons.py\",
>  line 225, in new_cached_exec\nExecute(command, user=exec_user)\n  File 
> \"/usr/lib/py
 thon2.6/site-packages/resource_management/core/base.py\", line 155, in 
__init__\nself.env.run()\n  File 
\"/usr/lib/python2.6/site-packages/resource_management/core/environment.py\", 
line 160, in run\nself.run_action(resource, action)\n  File 
\"/usr/lib/python2.6/site-packages/resource_management/core/environment.py\", 
line 124, in run_action\nprovider_action()\n  File 
\"/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py\",
 line 273, in action_run\ntries=self.resource.tries, 
try_sleep=self.resource.try_sleep)\n  File 
\"/usr/lib/python2.6/site-packages/resource_management/core/shell.py\", line 
70, in inner\nresult = function(command, **kwargs)\n  File 
\"/usr/lib/python2.6/site-packages/resource_management/core/shell.py\", line 
92, in checked_call\ntries=tries, try_sleep=try_sleep)\n  File 
\"/usr/lib/python2.6/site-packages/resource_management/core/shell.py\", line 
140, in _call_wrapper\nresult = _call(command, **kwargs_copy)\n  Fi
 le \"/usr/lib/python2.6/site-packages/resource_management/core/shell.py\", 
line 293, in _call\nraise 
Fail(err_msg)\nresource_management.core.exceptions.Fail: Execution of 
'/usr/bin/kinit -c /var/lib/ambari-agent/tmp/tmpGyLePc -kt 
/etc/security/keytabs/smokeuser.headless.keytab ambari...@example.com' returned 
1. kinit(v5): Internal credentials cache error when initializing cache 
/var/lib/ambari-agent/tmp/tmpGyLePc",
> ```
> 
> # Cause
> The MR service check was incorrectly calling `cached_kinit_executor` which is 
> used for internally testing keytab files in service status checks. This found 
> an issue where the user executing the check was not able to create the 
> temporary ticket cache due to ACLs on the relevant directory and created file 
> - created as the user executing ambari-agent, not the specified user. 
> 
> # Solution
> Ensured the user specified in `cached_kinit_executor` owns the temporary 
> ticket cache and correctly kinit-ed in the Map Reduce service check function.
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/security_commons.py
>  4ce2a5a 
>   
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/mapred_service_check.py
>  02fa57a 
>   
> ambari-server/src/test/python/stacks/2.0.6/YARN/test_mapreduce2_service_check.py
>  eef0faf 
> 
> Diff: https://reviews.apache.org/r/47729/diff/
> 
> 
> Testing
> ---
> 
> Manually tested
> 
> # Local test results: PASSED
> 
> # Jenkins test results: PENDING
> 
> 
> Thanks,
> 
> Robert Levas
> 
>

Review Request 47729: MAPREDUCE2_SERVICE_CHECK failed on secured cluster

2016-05-23 Thread Robert Levas


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47729/
---

Review request for Ambari, Jonathan Hurley, Nate Cole, and Sandor Magyari.


Bugs: AMBARI-16801
https://issues.apache.org/jira/browse/AMBARI-16801


Repository: ambari


Description
---

# Error
```
"stderr" : "Traceback (most recent call last):\n  File 
\"/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/mapred_service_check.py\",
 line 162, in \nMapReduce2ServiceCheck().execute()\n  File 
\"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py\",
 line 254, in execute\nmethod(env)\n  File 
\"/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/mapred_service_check.py\",
 line 140, in service_check\nparams.smokeuser_principal, params.hostname, 
params.tmp_dir)\n  File 
\"/usr/lib/python2.6/site-packages/resource_management/libraries/functions/security_commons.py\",
 line 204, in cached_kinit_executor\nnew_cached_exec(key, file_path, 
kinit_path, temp_dir, exec_user, keytab_file, principal, hostname)\n  File 
\"/usr/lib/python2.6/site-packages/resource_management/libraries/functions/security_commons.py\",
 line 225, in new_cached_exec\nExecute(command, user=exec_user)\n  File 
\"/usr/lib/pyth
 on2.6/site-packages/resource_management/core/base.py\", line 155, in 
__init__\nself.env.run()\n  File 
\"/usr/lib/python2.6/site-packages/resource_management/core/environment.py\", 
line 160, in run\nself.run_action(resource, action)\n  File 
\"/usr/lib/python2.6/site-packages/resource_management/core/environment.py\", 
line 124, in run_action\nprovider_action()\n  File 
\"/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py\",
 line 273, in action_run\ntries=self.resource.tries, 
try_sleep=self.resource.try_sleep)\n  File 
\"/usr/lib/python2.6/site-packages/resource_management/core/shell.py\", line 
70, in inner\nresult = function(command, **kwargs)\n  File 
\"/usr/lib/python2.6/site-packages/resource_management/core/shell.py\", line 
92, in checked_call\ntries=tries, try_sleep=try_sleep)\n  File 
\"/usr/lib/python2.6/site-packages/resource_management/core/shell.py\", line 
140, in _call_wrapper\nresult = _call(command, **kwargs_copy)\n  File
  \"/usr/lib/python2.6/site-packages/resource_management/core/shell.py\", line 
293, in _call\nraise 
Fail(err_msg)\nresource_management.core.exceptions.Fail: Execution of 
'/usr/bin/kinit -c /var/lib/ambari-agent/tmp/tmpGyLePc -kt 
/etc/security/keytabs/smokeuser.headless.keytab ambari...@example.com' returned 
1. kinit(v5): Internal credentials cache error when initializing cache 
/var/lib/ambari-agent/tmp/tmpGyLePc",
```

# Cause
The MR service check was incorrectly calling `cached_kinit_executor` which is 
used for internally testing keytab files in service status checks. This found 
an issue where the user executing the check was not able to create the 
temporary ticket cache due to ACLs on the relevant directory and created file - 
created as the user executing ambari-agent, not the specified user. 

# Solution
Ensured the user specified in `cached_kinit_executor` owns the temporary ticket 
cache and correctly kinit-ed in the Map Reduce service check function.


Diffs
-

  
ambari-common/src/main/python/resource_management/libraries/functions/security_commons.py
 4ce2a5a 
  
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/mapred_service_check.py
 02fa57a 
  
ambari-server/src/test/python/stacks/2.0.6/YARN/test_mapreduce2_service_check.py
 eef0faf 

Diff: https://reviews.apache.org/r/47729/diff/


Testing
---

Manually tested

# Local test results: PASSED

# Jenkins test results: PENDING


Thanks,

Robert Levas

Re: Review Request 47595: Implement in ams collector batch insert operations to ams-hbase

2016-05-23 Thread Myroslav Papirkovskyy


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47595/#review134361
---


Ship it!




Ship It!

- Myroslav Papirkovskyy


On Травень 23, 2016, 6:11 після полудня, Dmytro Sen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47595/
> ---
> 
> (Updated Травень 23, 2016, 6:11 після полудня)
> 
> 
> Review request for Ambari, Aravindan Vijayan and Sid Wagle.
> 
> 
> Bugs: AMBARI-16766
> https://issues.apache.org/jira/browse/AMBARI-16766
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Check if there is performance boost after replacing insert operations with 
> bulk loading. 
> If yes, implement and provide switch property for enabling/disabling this 
> feature.
> 
> 
> Diffs
> -
> 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java
>  83aa36b 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/MetricsCacheCommitterThread.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java
>  52ab083 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricConfiguration.java
>  8e9a8d6 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessorTest.java
>  290a98a 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/discovery/TestMetadataManager.java
>  92e4dfc 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml
>  e5758bf 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/service_check.py
>  8f369f7 
> 
> Diff: https://reviews.apache.org/r/47595/diff/
> 
> 
> Testing
> ---
> 
> Unit tests passed
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Re: Review Request 47710: No errors appeared after setting value with spaces to Atlas metadata_pid_dir

2016-05-23 Thread Vitalyi Brodetskyi


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47710/#review134357
---


Ship it!




Ship It!

- Vitalyi Brodetskyi


On Травень 23, 2016, 4:45 до полудня, Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47710/
> ---
> 
> (Updated Травень 23, 2016, 4:45 до полудня)
> 
> 
> Review request for Ambari and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-16808
> https://issues.apache.org/jira/browse/AMBARI-16808
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> STR:  
> 1) Install cluster, go to Customize Services page -> Atlas  
> 2) Set metadata_pid_dir to " /g ri d" for example (**starts with space** and
> **contains spaces**)
> 
> Expected: error message  
> Actually: no error message
> 
> Screenshot attached
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/configuration/atlas-env.xml
>  bae4de3 
> 
> Diff: https://reviews.apache.org/r/47710/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 47713: Agent host registration uses sudo even if sudo is not available.

2016-05-23 Thread Vitalyi Brodetskyi


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47713/#review134356
---


Ship it!




Ship It!

- Vitalyi Brodetskyi


On Травень 23, 2016, 5:54 до полудня, Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47713/
> ---
> 
> (Updated Травень 23, 2016, 5:54 до полудня)
> 
> 
> Review request for Ambari and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-16809
> https://issues.apache.org/jira/browse/AMBARI-16809
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/python/bootstrap.py 24755ae 
>   ambari-server/src/test/python/TestBootstrap.py 9888507 
> 
> Diff: https://reviews.apache.org/r/47713/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 47715: Ambari Agent security bypassed in Python=>2.7.9

2016-05-23 Thread Vitalyi Brodetskyi


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47715/#review134355
---


Ship it!




Ship It!

- Vitalyi Brodetskyi


On Травень 23, 2016, 6:46 до полудня, Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47715/
> ---
> 
> (Updated Травень 23, 2016, 6:46 до полудня)
> 
> 
> Review request for Ambari and Dmitro Lisnichenko.
> 
> 
> Bugs: AMBARI-16810
> https://issues.apache.org/jira/browse/AMBARI-16810
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> We hard-coded the Ambari Agents to ignore certification
> verification. But the reason why this was required was Python be un-secure by
> default:  
>   
> 
> 
> That method will cause signed certificates to not serve any purpose & is
> discouraged by RedHat & Python security experts:
> 
> > "It is also possible, though highly discouraged , to globally disable
> verification by monkeypatching the ssl module in versions of Python"
> 
> Instead we should abstract it to a setting (e.g. ssl_verify_cert) in the
> ambari-agent.ini such that users can turn certification verification if they
> provide a signed/trusted certificate.
> 
> 
> Diffs
> -
> 
>   ambari-agent/conf/unix/ambari-agent.ini 4ec16d6 
>   ambari-agent/src/main/python/ambari_agent/AmbariConfig.py f849fd1 
>   ambari-agent/src/main/python/ambari_agent/Controller.py aee0eec 
>   ambari-agent/src/main/python/ambari_agent/NetUtil.py 1d5cb29 
>   ambari-agent/src/main/python/ambari_agent/main.py 5340239 
> 
> Diff: https://reviews.apache.org/r/47715/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 47718: AMBARI-16811 : Kafka broker and RM start fail after Ambari upgrade while running JsonDecoder

2016-05-23 Thread Robert Levas


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47718/#review134354
---


Ship it!




Ship It!

- Robert Levas


On May 23, 2016, 9:58 a.m., Gautam Borad wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47718/
> ---
> 
> (Updated May 23, 2016, 9:58 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jaimin Jetly, Mahadev Konar, 
> Nate Cole, Robert Levas, Sumit Mohanty, and Velmurugan Periasamy.
> 
> 
> Bugs: AMBARI-16811
> https://issues.apache.org/jira/browse/AMBARI-16811
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Making proper stack support checks for repository creation during restart of 
> services after enabling plugin
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/ranger_functions_v2.py
>  a486da7 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/setup_ranger_plugin_xml.py
>  0fb017d 
> 
> Diff: https://reviews.apache.org/r/47718/diff/
> 
> 
> Testing
> ---
> 
> Verified :: 
> 
> Cluster was kerberised successfully and services were getting re-started, 
> after enabling Kerberos from Ambari. Kafka and Resource manager restarts 
> worked even after ambari upgrade from 2.2 to 2.4.
> 
> 
> Thanks,
> 
> Gautam Borad
> 
>

Re: Review Request 47718: AMBARI-16811 : Kafka broker and RM start fail after Ambari upgrade while running JsonDecoder

2016-05-23 Thread Nate Cole


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47718/#review134352
---


Ship it!




Ship It!

- Nate Cole


On May 23, 2016, 9:58 a.m., Gautam Borad wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47718/
> ---
> 
> (Updated May 23, 2016, 9:58 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jaimin Jetly, Mahadev Konar, 
> Nate Cole, Robert Levas, Sumit Mohanty, and Velmurugan Periasamy.
> 
> 
> Bugs: AMBARI-16811
> https://issues.apache.org/jira/browse/AMBARI-16811
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Making proper stack support checks for repository creation during restart of 
> services after enabling plugin
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/ranger_functions_v2.py
>  a486da7 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/setup_ranger_plugin_xml.py
>  0fb017d 
> 
> Diff: https://reviews.apache.org/r/47718/diff/
> 
> 
> Testing
> ---
> 
> Verified :: 
> 
> Cluster was kerberised successfully and services were getting re-started, 
> after enabling Kerberos from Ambari. Kafka and Resource manager restarts 
> worked even after ambari upgrade from 2.2 to 2.4.
> 
> 
> Thanks,
> 
> Gautam Borad
> 
>

Re: Review Request 47701: AMBARI-16642: After enabling HTTPS for YARN, NodeManager JVM Metrics on YARN Heatmaps show NA.

2016-05-23 Thread Qin Liu



> On May 23, 2016, 9:09 a.m., Dmytro Sen wrote:
> > Ship It!

Dmytro, thank you so much for your quick response!  I really appreciate it.


- Qin


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47701/#review134322
---


On May 23, 2016, 5:59 a.m., Qin Liu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47701/
> ---
> 
> (Updated May 23, 2016, 5:59 a.m.)
> 
> 
> Review request for Ambari, Dmytro Sen and Sid Wagle.
> 
> 
> Bugs: AMBARI-16642
> https://issues.apache.org/jira/browse/AMBARI-16642
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> After enabling HTTPS for YARN, "NodeManager GC Time" and "NodeManager JVM 
> Heap Memory Used" widgets on YARN Heatmaps show NA. The root cause is that 
> yarn.nodemanager.webapp.address rather than 
> yarn.nodemanager.webapp.https.address is used to get JMX metrics. 
> 
> Steps to reproduce:
> 1. install a cluster with default from Ambari Web UI.
> 2. configuring SSL for HDFS, YARN, and MapReduce.
> 3. enable HTTPS for YARN
> 1). Set the following property in Advanced yarn-site from Ambari Web UI:
> yarn.http.policy=HTTPS_ONLY
> 2). Add the following property in Custom yarn-site from Ambari Web UI:
> yarn.nodemanager.webapp.https.address=0.0.0.0:8044
> 4. "NodeManager GC Time" and "NodeManager JVM Heap Memory Used" widgets on 
> YARN Heatmaps will show NA.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractProviderModule.java
>  36be39e 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/JMXHostProviderTest.java
>  a6e5501 
> 
> Diff: https://reviews.apache.org/r/47701/diff/
> 
> 
> Testing
> ---
> 
> 1. The fix has been tested with HDP UI
> "NodeManager GC Time" and "NodeManager JVM Heap Memory Used" metrics show up 
> after apply the fix.
> 
> 2. JMXHostProviderTest.java has been modified to verify JMX protocol and port 
> for NodeManager.
> Test set: org.apache.ambari.server.controller.internal.JMXHostProviderTest
> ---
> Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 120.274 sec - 
> in org.apache.ambari.server.controller.internal.JMXHostProviderTest
> 
> 3. The latest result of running ambari-server tests:
> Results :
> Failed tests: 
>   DataStoreImplTest.testFind:409 
>   Unexpected method call EntityManager.getDelegate():
> EntityManager.find(capture(Nothing captured yet), 99): expected: 1, 
> actual: 0
> EntityManager.close(): expected: 1, actual: 0
>   DataStoreImplTest.testFindAll:453 
>   Unexpected method call EntityManager.getDelegate():
> EntityManager.createQuery("SELECT e FROM 
> DS_DataStoreImplTest$TestEntity_1 e WHERE e.DS_id=99"): expected: 1, actual: 0
> EntityManager.close(): expected: 1, actual: 0
>   DataStoreImplTest.testFindAll_multiple:518 
>   Unexpected method call EntityManager.getDelegate():
> EntityManager.createQuery("SELECT e FROM 
> DS_DataStoreImplTest$TestEntity_1 e WHERE e.DS_name='foo'"): expected: 1, 
> actual: 0
> EntityManager.close(): expected: 1, actual: 0
>   DataStoreImplTest.testRemove:370 
>   Unexpected method call EntityManager.getDelegate():
> EntityManager.getReference(capture(Nothing captured yet), 99): expected: 
> 1, actual: 0
> EntityManager.remove(EasyMock for interface 
> org.eclipse.persistence.dynamic.DynamicEntity): expected: 1, actual: 0
> EntityManager.close(): expected: 1, actual: 0
>   DataStoreImplTest.testStore_create:128 
>   Unexpected method call EntityManager.getDelegate():
> EntityManager.find(capture(Nothing captured yet), "bar"): expected: 1, 
> actual: 0
> EntityManager.find(capture(Nothing captured yet), 99): expected: 1, 
> actual: 0
> EntityManager.persist(capture(Nothing captured yet)): expected: 1, 
> actual: 0
> EntityManager.persist(capture(Nothing captured yet)): expected: 1, 
> actual: 0
> EntityManager.close(): expected: 1, actual: 0
>   DataStoreImplTest.testStore_create_largeEntity:222 
>   Unexpected method call EntityManager.getDelegate():
> EntityManager.close(): expected: 1, actual: 0
>   DataStoreImplTest.testStore_create_longStringValue:182 
>   Unexpected method call EntityManager.getDelegate():
> EntityManager.find(capture(Nothing captured yet), 99): expected: 1, 
> actual: 0
> EntityManager.close(): expected: 1, actual: 0
>   DataStoreImplTest.testStore_update:276 
>   Unexpected method call EntityManager.getDelegate():
> EntityManager.find(capture(Nothing captured yet), "bar"): expected: 1, 
> actual: 0
> EntityManager.find(capture(Nothing captured yet), 99): expected:

Re: Review Request 47726: Blueprint Export does not replace hive.llap.zk.sm.connectionString

2016-05-23 Thread Sebastian Toader


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47726/#review134349
---


Fix it, then Ship it!





ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
 (line 2395)


Add unit test to ensure that we catch if this is removed accidentally.


- Sebastian Toader


On May 23, 2016, 4:07 p.m., Daniel Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47726/
> ---
> 
> (Updated May 23, 2016, 4:07 p.m.)
> 
> 
> Review request for Ambari, Balázs Bence Sári, Laszlo Puskas, Oliver Szabo, 
> Robert Nettleton, Sandor Magyari, Sumit Mohanty, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-16820
> https://issues.apache.org/jira/browse/AMBARI-16820
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> MultiHostUpdater was not assigned to the property 
> hive.llap.zk.sm.connectionString
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
>  9cabdbb 
> 
> Diff: https://reviews.apache.org/r/47726/diff/
> 
> 
> Testing
> ---
> 
> In progress...
> 
> 
> Thanks,
> 
> Daniel Gergely
> 
>

Review Request 47726: Blueprint Export does not replace hive.llap.zk.sm.connectionString

2016-05-23 Thread Daniel Gergely


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47726/
---

Review request for Ambari, Balázs Bence Sári, Laszlo Puskas, Oliver Szabo, 
Sandor Magyari, Sumit Mohanty, and Sebastian Toader.


Bugs: AMBARI-16820
https://issues.apache.org/jira/browse/AMBARI-16820


Repository: ambari


Description
---

MultiHostUpdater was not assigned to the property 
hive.llap.zk.sm.connectionString


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
 9cabdbb 

Diff: https://reviews.apache.org/r/47726/diff/


Testing
---

In progress...


Thanks,

Daniel Gergely

Review Request 47718: AMBARI-16811 : Kafka broker and RM start fail after Ambari upgrade while running JsonDecoder

2016-05-23 Thread Gautam Borad


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47718/
---

Review request for Ambari, Alejandro Fernandez, Jaimin Jetly, Mahadev Konar, 
Nate Cole, Robert Levas, Sumit Mohanty, and Velmurugan Periasamy.


Bugs: AMBARI-16811
https://issues.apache.org/jira/browse/AMBARI-16811


Repository: ambari


Description
---

Making proper stack support checks for repository creation during restart of 
services after enabling plugin


Diffs
-

  
ambari-common/src/main/python/resource_management/libraries/functions/ranger_functions_v2.py
 a486da7 
  
ambari-common/src/main/python/resource_management/libraries/functions/setup_ranger_plugin_xml.py
 0fb017d 

Diff: https://reviews.apache.org/r/47718/diff/


Testing
---

Verified :: 

Cluster was kerberised successfully and services were getting re-started, after 
enabling Kerberos from Ambari. Kafka and Resource manager restarts worked even 
after ambari upgrade from 2.2 to 2.4.


Thanks,

Gautam Borad

Re: Review Request 47631: 'Configure Ambari Identity' fails when enabling Kerberos on non-root Ambari server

2016-05-23 Thread Sandor Magyari


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47631/#review134344
---


Ship it!




Ship It!

- Sandor Magyari


On May 20, 2016, 2:10 p.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47631/
> ---
> 
> (Updated May 20, 2016, 2:10 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Nate Cole, Sandor Magyari, and 
> Sumit Mohanty.
> 
> 
> Bugs: AMBARI-16787
> https://issues.apache.org/jira/browse/AMBARI-16787
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Configure Ambari Identity phase of Enable Kerberos Wizard fails since Ambari 
> does not have permission perform some root-level file system tasks like 
> chown. 
> 
> STR:
> 1. Install Ambari, executing as non-root
> 2. Create cluster (Zookeeper-only is fine)
> 3. Enable Kerberos (any KDC is fine)
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelperImpl.java
>  15bd2d9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/ConfigureAmbariIndetityServerAction.java
>  bbc7246 
>   
> ambari-server/src/main/java/org/apache/ambari/server/utils/ShellCommandUtil.java
>  b5d2b70 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/kerberos.json c5ebe20 
>   
> ambari-server/src/test/java/org/apache/ambari/server/utils/TestShellCommandUtil.java
>  fc27eda 
> 
> Diff: https://reviews.apache.org/r/47631/diff/
> 
> 
> Testing
> ---
> 
> Manually tested in cluster using UI and using Blueprints where Ambari was 
> executed as root and non-root users.
> 
> # Local test results: 
> 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 1:12:58.862s
> [INFO] Finished at: Thu May 19 23:23:12 EDT 2016
> [INFO] Final Memory: 59M/1410M
> [INFO] 
> 
> 
> 
> # Jenkins test results: PENDING
> 
> 
> Thanks,
> 
> Robert Levas
> 
>

Re: Review Request 47724: AMBARI-16818 Ambari Quick link missing from Zeppelin

2016-05-23 Thread Renjith Kamath


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47724/
---

(Updated May 23, 2016, 1:41 p.m.)


Review request for Ambari, Alejandro Fernandez, DIPAYAN BHOWMICK, Jayush 
Luniya, Pallav Kulshreshtha, Rohit Choudhary, and Sumit Mohanty.


Summary (updated)
-

AMBARI-16818 Ambari Quick link missing from Zeppelin


Bugs: AMBARI-16818
https://issues.apache.org/jira/browse/AMBARI-16818


Repository: ambari


Description
---

Ambari Quick link missing from Zeppelin page in Ambari.
The quick link should link to install Zeppelin host:port like it does for other 
components.


Diffs
-

  
ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/metainfo.xml
 1b760ac 
  
ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/quicklinks/quicklinks.json
 PRE-CREATION 

Diff: https://reviews.apache.org/r/47724/diff/


Testing
---


Thanks,

Renjith Kamath

Re: Review Request 47631: 'Configure Ambari Identity' fails when enabling Kerberos on non-root Ambari server

2016-05-23 Thread Robert Levas


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47631/#review134343
---



Any other reviewers want to take a shot? :)

jhurley, smagyari?

- Robert Levas


On May 20, 2016, 10:10 a.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47631/
> ---
> 
> (Updated May 20, 2016, 10:10 a.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Nate Cole, Sandor Magyari, and 
> Sumit Mohanty.
> 
> 
> Bugs: AMBARI-16787
> https://issues.apache.org/jira/browse/AMBARI-16787
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Configure Ambari Identity phase of Enable Kerberos Wizard fails since Ambari 
> does not have permission perform some root-level file system tasks like 
> chown. 
> 
> STR:
> 1. Install Ambari, executing as non-root
> 2. Create cluster (Zookeeper-only is fine)
> 3. Enable Kerberos (any KDC is fine)
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelperImpl.java
>  15bd2d9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/ConfigureAmbariIndetityServerAction.java
>  bbc7246 
>   
> ambari-server/src/main/java/org/apache/ambari/server/utils/ShellCommandUtil.java
>  b5d2b70 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/kerberos.json c5ebe20 
>   
> ambari-server/src/test/java/org/apache/ambari/server/utils/TestShellCommandUtil.java
>  fc27eda 
> 
> Diff: https://reviews.apache.org/r/47631/diff/
> 
> 
> Testing
> ---
> 
> Manually tested in cluster using UI and using Blueprints where Ambari was 
> executed as root and non-root users.
> 
> # Local test results: 
> 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 1:12:58.862s
> [INFO] Finished at: Thu May 19 23:23:12 EDT 2016
> [INFO] Final Memory: 59M/1410M
> [INFO] 
> 
> 
> 
> # Jenkins test results: PENDING
> 
> 
> Thanks,
> 
> Robert Levas
> 
>

Re: Review Request 47018: [preview] "ambari-server upgrade" shouldn't automatically add stack configs

2016-05-23 Thread Nate Cole



> On May 19, 2016, 10:20 a.m., Nate Cole wrote:
> > ambari-server/src/main/resources/property-schema.xsd, lines 68-70
> > 
> >
> > All 3 required?
> 
> Dmitro Lisnichenko wrote:
> Yes, these are mandatory attributes for both on-ambari-upgrade and 
> on-stack-upgrade tags

Ok, just confirming.


- Nate


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47018/#review133941
---


On May 18, 2016, 1:11 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47018/
> ---
> 
> (Updated May 18, 2016, 1:11 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, 
> and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-16272
> https://issues.apache.org/jira/browse/AMBARI-16272
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Today, "ambari-server upgrade" will automatically add stack configs.
> However, it also causes problems when default properties or properties with 
> default value such as "localhost" end up being added.
> 
> This led to many bugs. E.g., cluster with NameNode HA shouldn't automatically 
> add dfs.namenode.secondary.http-address
> 
> This logic today will even add new config types. E.g., add ranger-env even 
> though Ranger is not installed. If the customer then upgrades the stack from 
> HDP 2.2 to 2.3, and then adds Ranger, they can get the wrong configs.
> If we change this behavior, it's good to do so in a major release such as 2.4
> 
> We add required xml tags/attributes to properties:
> 
>   prop_name
>   prop_val
>   
>   
> 
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java
>  fb3ae69 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackManager.java 
> 8a352bd 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 
> 854dd0b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/PropertyInfo.java 
> 34b3ba1 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/PropertyUpgradeBehavior.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java
>  2e857ed 
>   ambari-server/src/main/resources/property-schema.xsd PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderHDP22Test.java
>  a4a3108 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderTest.java
>  8f53f6a 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/PropertyInfoTest.java
>  b11c5d8 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/ServicePropertiesTest.java
>  PRE-CREATION 
>   ambari-server/src/test/resources/propertiesFile.xsd PRE-CREATION 
>   script.sh PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/47018/diff/
> 
> 
> Testing
> ---
> 
> patch preview for now. Feedback is welcome
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Re: Review Request 47321: Ambari to download repo file besides creating it.

2016-05-23 Thread Nate Cole


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47321/#review134338
---



Was this effort abandoned?  If so, the review should get closed.

- Nate Cole


On May 12, 2016, 12:13 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47321/
> ---
> 
> (Updated May 12, 2016, 12:13 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Dmitro Lisnichenko, Jonathan 
> Hurley, and Nate Cole.
> 
> 
> Bugs: AMBARI-16639
> https://issues.apache.org/jira/browse/AMBARI-16639
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Currently Ambari only takes a base url and creates a HDP file.
> 
> In addition to this, if ambari can take a repo url and download it, we will
> have the flexibility of combining 2 or more repos and downloading artifacts
> from multiple repositories.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/test/python/resource_management/TestRepositoryResource.py 
> a69b57b 
>   ambari-common/src/main/python/resource_management/core/base.py 1500e1f 
>   
> ambari-common/src/main/python/resource_management/libraries/providers/repository.py
>  f476e29 
>   
> ambari-common/src/main/python/resource_management/libraries/resources/repository.py
>  2484d67 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java
>  77f2ede 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariActionExecutionHelper.java
>  bb58670 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java
>  5e51f5f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  7884d02 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/RepositoryRequest.java
>  84929bf 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/RepositoryResponse.java
>  30bd0db 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RepositoryResourceProvider.java
>  3b5b0a7 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RepositoryVersionResourceProvider.java
>  57fb115 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/utilities/RepositoryHelper.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RepositoryEntity.java
>  49d53a9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackContext.java 
> da7f021 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackManager.java 
> 8a352bd 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
> 6129ec0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/RepositoryDistributionType.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/RepositoryInfo.java
>  29776ed 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/LatestRepoCallable.java
>  fc2bab1 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/RepositoryXml.java
>  69eb39d 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelper.java
>  50d6028 
>   ambari-server/src/main/resources/custom_actions/scripts/install_packages.py 
> 3133984 
>   ambari-server/src/main/resources/custom_actions/scripts/update_repo.py 
> 9f2107d 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/repo_initialization.py
>  a35dce7 
>   ambari-server/src/main/resources/version_definition.xsd b6360ee 
>   
> ambari-server/src/test/java/org/apache/ambari/server/api/services/AmbariMetaInfoTest.java
>  e5bb7ca 
>   
> ambari-server/src/test/java/org/apache/ambari/server/api/services/KerberosServiceMetaInfoTest.java
>  befd594 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackManagerCommonServicesTest.java
>  6f3cf78 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackManagerMiscTest.java
>  dda1e7a 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackManagerTest.java
>  8e9f81a 
>   ambari-server/src/test/python/custom_actions/TestInstallPackages.py 77e3bee 
>   
> ambari-server/src/test/python/stacks/2.0.6/hooks/before-INSTALL/test_before_install.py
>  082d200 
> 
> Diff: https://reviews.apache.org/r/47321/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 45169: AMBARI-15388 - Upgrade XML should be pushed down as much as possible to the services

2016-05-23 Thread Nate Cole


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45169/#review134336
---



It looks like this was pushed, is there a need to keep the review open?

- Nate Cole


On May 19, 2016, 11:23 a.m., Tim Thorpe wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45169/
> ---
> 
> (Updated May 19, 2016, 11:23 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Jayush 
> Luniya, and Nate Cole.
> 
> 
> Bugs: AMBARI-15388
> https://issues.apache.org/jira/browse/AMBARI-15388
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Currently the upgrade is defined as a series of xml files specific to the 
> current stack version and the target stack version. Each upgrade xml defines 
> the overall sequence of the upgrade and what needs to be done for each 
> service. It would both easier to maintain and easier to add new services, if 
> the services themselves could specify what should be done during their 
> upgrade.
> 
> There are two ways to make these changes, the alternate approach would be to 
> only make the java changes and not split the upgrade xml files.  This would 
> still allow new services to add themselves into the upgrade.  The benefit of 
> this is that for the stack services you only have one upgrade xml file.  The 
> problem with that is it is easier for a particular service to have 
> unintentional changes between upgrade xml files.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/CommonServiceDirectory.java
>  7f7a49e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceDirectory.java
>  8a7b42b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java 
> f781574 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java
>  13d5047 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
> 5a18b3f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java
>  88f6e19 
>   ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java 
> 43cefb9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/UpgradePack.java
>  b860731 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ClusterGrouping.java
>  3325469 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/Grouping.java
>  67d7fdb 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ServiceCheckGrouping.java
>  5cda422 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.3.xml 
> 6b74af0 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.4.xml 
> 9fb2bba 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> 1e040e6 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.4.xml 
> e3bc7a3 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> 6e27da6 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.5.xml 
> d755516 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackManagerMiscTest.java
>  dda1e7a 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/stack/UpgradePackTest.java
>  15be8b4 
>   
> ambari-server/src/test/resources/stacks/HDP/2.0.5/services/HDFS/upgrades/HDP/2.2.0/upgrade_test_15388.xml
>  PRE-CREATION 
>   
> ambari-server/src/test/resources/stacks/HDP/2.2.0/upgrades/upgrade_test_15388.xml
>  PRE-CREATION 
>   
> ambari-server/src/test/resources/stacks_with_upgrade_cycle/HDP/2.2.0/metainfo.xml
>  PRE-CREATION 
>   
> ambari-server/src/test/resources/stacks_with_upgrade_cycle/HDP/2.2.0/repos/hdp.json
>  PRE-CREATION 
>   
> ambari-server/src/test/resources/stacks_with_upgrade_cycle/HDP/2.2.0/repos/repoinfo.xml
>  PRE-CREATION 
>   
> ambari-server/src/test/resources/stacks_with_upgrade_cycle/HDP/2.2.0/repos/version-2.2.0.4-123.xml
>  PRE-CREATION 
>   
> ambari-server/src/test/resources/stacks_with_upgrade_cycle/HDP/2.2.0/role_command_order.json
>  PRE-CREATION 
>   
> ambari-server/src/test/resources/stacks_with_upgrade_cycle/HDP/2.2.0/services/HDFS/metainfo.xml
>  PRE-CREATION 
>   
> ambari-server/src/test/resources/stacks_with_upgrade_cycle/HDP/2.2.0/services/HDFS/upgrades/HDP/2.2.0/upgrade_test_15388.xml
>  PRE-CREATION 
>   
> ambari-server/src/test/resources/stacks_with_upgrade_cycle/HDP/2.2.0/upgrades/config-upgrade.xml
>  PRE-CREATION 
>   
> ambari-server/src/test/resources/stacks_with_upgrade_cycle/HDP/2.2.0/upgrades/upgrade_test_15388.xml
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/45169/diff/
> 
> 
>

Re: Review Request 46270: AMBARI-15917 : Add ability to specify a custom trust store for ranger admin via ambari config

2016-05-23 Thread Nate Cole


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46270/#review134335
---



What is the status of this review?  It looks like it was pushed and can be 
closed?

- Nate Cole


On May 4, 2016, 3:12 a.m., Gautam Borad wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46270/
> ---
> 
> (Updated May 4, 2016, 3:12 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jaimin Jetly, Mahadev Konar, 
> Nate Cole, Sumit Mohanty, and Velmurugan Periasamy.
> 
> 
> Bugs: AMBARI-15917
> https://issues.apache.org/jira/browse/AMBARI-15917
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Expose custom trust store related configs for Ranger on Ambari UI.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/RANGER/configuration/ranger-admin-site.xml
>  d533dbc 
> 
> Diff: https://reviews.apache.org/r/46270/diff/
> 
> 
> Testing
> ---
> 
> Verified addition of : ranger.truststore.file and ranger.truststore.password 
> in ranger-admin-site.xml.
> 
> 
> Thanks,
> 
> Gautam Borad
> 
>

Re: Review Request 47721: Editing local cluster name breaks all view instances

2016-05-23 Thread Rohit Choudhary


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47721/#review134334
---


Ship it!




Ship It!

- Rohit Choudhary


On May 23, 2016, 10:21 a.m., Gaurav Nagar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47721/
> ---
> 
> (Updated May 23, 2016, 10:21 a.m.)
> 
> 
> Review request for Ambari, DIPAYAN BHOWMICK, Nitiraj Rathore, Pallav 
> Kulshreshtha, Rohit Choudhary, and Ashwin Rajeev.
> 
> 
> Bugs: AMBARI-16812
> https://issues.apache.org/jira/browse/AMBARI-16812
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Changed cluster_handle in view instance to have cluster_id instead of 
> cluster_name.
> Made Remote cluster name editable
> 
> 
> Diffs
> -
> 
>   
> ambari-admin/src/main/resources/ui/admin-web/app/scripts/controllers/ambariViews/CreateViewInstanceCtrl.js
>  034b113 
>   
> ambari-admin/src/main/resources/ui/admin-web/app/scripts/controllers/ambariViews/ViewsEditCtrl.js
>  2b1fb3b 
>   
> ambari-admin/src/main/resources/ui/admin-web/app/scripts/controllers/remoteClusters/RemoteClustersEditCtrl.js
>  20eb552 
>   
> ambari-admin/src/main/resources/ui/admin-web/app/scripts/services/Cluster.js 
> 0bf73d5 
>   
> ambari-admin/src/main/resources/ui/admin-web/app/scripts/services/RemoteCluster.js
>  e4826e6 
>   ambari-admin/src/main/resources/ui/admin-web/app/scripts/services/View.js 
> dfe7ba8 
>   
> ambari-admin/src/main/resources/ui/admin-web/app/views/ambariViews/create.html
>  051d720 
>   
> ambari-admin/src/main/resources/ui/admin-web/app/views/ambariViews/edit.html 
> 3a69a50 
>   
> ambari-admin/src/main/resources/ui/admin-web/app/views/remoteClusters/editRemoteClusterPage.html
>  b5890bf 
>   
> ambari-admin/src/main/resources/ui/admin-web/test/unit/controllers/CreateViewInstanceCtrl.js
>  595fcc0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/ClusterNotFoundException.java
>  cd338a0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RemoteClusterResourceProvider.java
>  413dbff 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ViewInstanceResourceProvider.java
>  b8ed215 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/RemoteAmbariClusterDAO.java
>  72ab368 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RemoteAmbariClusterEntity.java
>  99c9f2a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/ViewInstanceEntity.java
>  98bafdd 
>   ambari-server/src/main/java/org/apache/ambari/server/state/Clusters.java 
> e660d20 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClustersImpl.java
>  f78dd95 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
>  41f538e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/view/RemoteAmbariClusterRegistry.java
>  38a47a4 
>   
> ambari-server/src/main/java/org/apache/ambari/server/view/ViewContextImpl.java
>  e98a4cd 
>   ambari-server/src/main/java/org/apache/ambari/server/view/ViewRegistry.java 
> 6d4ef82 
>   ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql 31dd8f2 
>   ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 3c4f1ca 
>   ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql f31af16 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql d1147b7 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql 
> 5a7a6a5 
>   ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 2dba3b7 
>   ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql e769e5b 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UserPrivilegeResourceProviderTest.java
>  88b00ca 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ViewInstanceResourceProviderTest.java
>  1044570 
>   
> ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AuthorizationHelperTest.java
>  9c59aab 
>   
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog240Test.java
>  20fa50f 
>   
> ambari-views/src/main/java/org/apache/ambari/view/ViewInstanceDefinition.java 
> cea4c88 
>   
> contrib/views/hive/src/main/java/org/apache/ambari/view/hive/PropertyValidator.java
>  ae73bc0 
>   
> contrib/views/pig/src/main/java/org/apache/ambari/view/pig/PropertyValidator.java
>  d3c9866 
> 
> Diff: https://reviews.apache.org/r/47721/diff/
> 
> 
> Testing
> ---
> 
> Manually tested
> 
> 
> Thanks,
> 
> Gaurav Nagar
> 
>

Re: Review Request 47656: AMBARI-12885 - Dynamic stack extensions - install and upgrade support for custom services

2016-05-23 Thread Alexander Denissov


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47656/#review134330
---




ambari-server/src/main/java/org/apache/ambari/server/api/resources/ExtensionLinkResourceDefinition.java
 (line 51)


are these supposed to be commented out ? if yes, maybe remove them ?



ambari-server/src/main/java/org/apache/ambari/server/controller/internal/Extension.java
 (line 305)


a lot of commented out code here


- Alexander Denissov


On May 20, 2016, 7:03 p.m., Tim Thorpe wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47656/
> ---
> 
> (Updated May 20, 2016, 7:03 p.m.)
> 
> 
> Review request for Ambari, Alexander Denissov, Alejandro Fernandez, bhuvnesh 
> chaudhary, Jayush Luniya, Oleksandr Diachenko, Sumit Mohanty, Srimanth 
> Gunturi, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-12885
> https://issues.apache.org/jira/browse/AMBARI-12885
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> The purpose of this proposal is to facilitate adding custom services to an 
> existing stack. Ideally this would support adding and upgrading custom 
> services separately from the core services defined in the stack. In 
> particular we are looking at custom services that need to support several 
> different stacks (different distributions of Ambari). The release cycle of 
> the custom services may be different from that of the core stack; that is, a 
> custom service may be upgraded at a different rate than the core distribution 
> itself and may be upgraded multiple times within the lifespan of a single 
> release of the core distribution.
> 
> One possible approach to handling this would be dynamically extending a stack 
> (after install time). It would be best to extend the stack in packages where 
> a stack extension package can have one or more custom services.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py 
> fcea23f 
>   ambari-agent/src/main/python/ambari_agent/FileCache.py b7c5dee 
>   ambari-server/conf/unix/ambari.properties 9f1692e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/ExtensionLinkResourceDefinition.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/ExtensionResourceDefinition.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/ExtensionVersionResourceDefinition.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/ResourceInstanceFactoryImpl.java
>  9c864b6 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java
>  c54fe3f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/ExtensionLinksService.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/ExtensionsService.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/StacksService.java
>  557ce98 
>   
> ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
>  f5e8f39 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementController.java
>  d6b9d0e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  dd3d098 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/ExtensionLinkRequest.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/ExtensionLinkResponse.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/ExtensionRequest.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/ExtensionResponse.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/ExtensionVersionRequest.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/ExtensionVersionResponse.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractControllerResourceProvider.java
>  3721113 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/Extension.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ExtensionLinkResourceProvider.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ExtensionResourceProvider.java
>  PRE-CREATION 
>   
>

Re: Review Request 47634: AMBARI-16788: Add HDFS keytab on RegionServer node

2016-05-23 Thread Robert Levas


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47634/#review134331
---


Ship it!




Ship It!

- Robert Levas


On May 23, 2016, 6:09 a.m., Gautam Borad wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47634/
> ---
> 
> (Updated May 23, 2016, 6:09 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Robert Levas, Sumit Mohanty, 
> and Velmurugan Periasamy.
> 
> 
> Bugs: AMBARI-16788
> https://issues.apache.org/jira/browse/AMBARI-16788
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Currently HDFS keytab is not distributed to HBASE RegionServer node. In such 
> case Region Server restart will fail, since it wont find the keytab.
> The keytab is required to create HDFS dir to store audit logs to.
> 
> Added the keytab in kerberos.json file.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/hbase_master.py
>  840411f 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/hbase_regionserver.py
>  4001912 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/setup_ranger_hbase.py
>  abb47e5 
> 
> Diff: https://reviews.apache.org/r/47634/diff/
> 
> 
> Testing
> ---
> 
> Tested on 3 node cluster and ensured that the keytab is available on 
> RegionServer node.
> 
> 
> Thanks,
> 
> Gautam Borad
> 
>

Review Request 47721: Editing local cluster name breaks all view instances

2016-05-23 Thread Gaurav Nagar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47721/
---

Review request for Ambari, DIPAYAN BHOWMICK, Nitiraj Rathore, Pallav 
Kulshreshtha, Rohit Choudhary, and Ashwin Rajeev.


Bugs: AMBARI-16812
https://issues.apache.org/jira/browse/AMBARI-16812


Repository: ambari


Description
---

Changed cluster_handle in view instance to have cluster_id instead of 
cluster_name.
Made Remote cluster name editable


Diffs
-

  
ambari-admin/src/main/resources/ui/admin-web/app/scripts/controllers/ambariViews/CreateViewInstanceCtrl.js
 034b113 
  
ambari-admin/src/main/resources/ui/admin-web/app/scripts/controllers/ambariViews/ViewsEditCtrl.js
 2b1fb3b 
  
ambari-admin/src/main/resources/ui/admin-web/app/scripts/controllers/remoteClusters/RemoteClustersEditCtrl.js
 20eb552 
  ambari-admin/src/main/resources/ui/admin-web/app/scripts/services/Cluster.js 
0bf73d5 
  
ambari-admin/src/main/resources/ui/admin-web/app/scripts/services/RemoteCluster.js
 e4826e6 
  ambari-admin/src/main/resources/ui/admin-web/app/scripts/services/View.js 
dfe7ba8 
  
ambari-admin/src/main/resources/ui/admin-web/app/views/ambariViews/create.html 
051d720 
  ambari-admin/src/main/resources/ui/admin-web/app/views/ambariViews/edit.html 
3a69a50 
  
ambari-admin/src/main/resources/ui/admin-web/app/views/remoteClusters/editRemoteClusterPage.html
 b5890bf 
  
ambari-admin/src/main/resources/ui/admin-web/test/unit/controllers/CreateViewInstanceCtrl.js
 595fcc0 
  
ambari-server/src/main/java/org/apache/ambari/server/ClusterNotFoundException.java
 cd338a0 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RemoteClusterResourceProvider.java
 413dbff 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ViewInstanceResourceProvider.java
 b8ed215 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/dao/RemoteAmbariClusterDAO.java
 72ab368 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RemoteAmbariClusterEntity.java
 99c9f2a 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/ViewInstanceEntity.java
 98bafdd 
  ambari-server/src/main/java/org/apache/ambari/server/state/Clusters.java 
e660d20 
  
ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClustersImpl.java
 f78dd95 
  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
 41f538e 
  
ambari-server/src/main/java/org/apache/ambari/server/view/RemoteAmbariClusterRegistry.java
 38a47a4 
  
ambari-server/src/main/java/org/apache/ambari/server/view/ViewContextImpl.java 
e98a4cd 
  ambari-server/src/main/java/org/apache/ambari/server/view/ViewRegistry.java 
6d4ef82 
  ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql 31dd8f2 
  ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 3c4f1ca 
  ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql f31af16 
  ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql d1147b7 
  ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql 
5a7a6a5 
  ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 2dba3b7 
  ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql e769e5b 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UserPrivilegeResourceProviderTest.java
 88b00ca 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ViewInstanceResourceProviderTest.java
 1044570 
  
ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AuthorizationHelperTest.java
 9c59aab 
  
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog240Test.java
 20fa50f 
  ambari-views/src/main/java/org/apache/ambari/view/ViewInstanceDefinition.java 
cea4c88 
  
contrib/views/hive/src/main/java/org/apache/ambari/view/hive/PropertyValidator.java
 ae73bc0 
  
contrib/views/pig/src/main/java/org/apache/ambari/view/pig/PropertyValidator.java
 d3c9866 

Diff: https://reviews.apache.org/r/47721/diff/


Testing
---

Manually tested


Thanks,

Gaurav Nagar

Review Request 47722: AMBARI-16813 Ranger Usersync config to support Group Based Search for LDAP Sync Source

2016-05-23 Thread Mugdha Varadkar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47722/
---

Review request for Ambari, Alejandro Fernandez, Andrew Onischuk, Gautam Borad, 
Jaimin Jetly, Jayush Luniya, and Velmurugan Periasamy.


Bugs: AMBARI-16813
https://issues.apache.org/jira/browse/AMBARI-16813


Repository: ambari


Description
---

Changes include
- Added two new properties under ranger-ugsync-site.xml to support Group based 
search.
- quicklinks.json
- adding multiLine for property 
hadoop.kms.authentication.kerberos.name.rules
- hdfs audit for KMS should pick up the fs.defaultFS from core-site.xml
- Add Proxy in kms-site.xml, hadoop.kms.proxyuser.ranger.users/groups/hosts


Diffs
-

  
ambari-common/src/main/python/resource_management/libraries/functions/package_conditions.py
 e9c9507 
  
ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/params.py
 d4418e0 
  
ambari-server/src/main/resources/common-services/RANGER/0.5.0/quicklinks/quicklinks.json
 795e464 
  
ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-ugsync-site.xml
 5298169 
  
ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/tagsync-application-properties.xml
 ea00fbe 
  
ambari-server/src/main/resources/common-services/RANGER/0.6.0/themes/theme_version_3.json
 d5efca4 
  
ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/configuration/kms-site.xml
 e559d92 
  
ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/configuration/ranger-kms-audit.xml
 7c98fb3 
  ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 
188479e 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/RANGER_KMS/configuration/dbks-site.xml
 fca42b3 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/RANGER_KMS/themes/theme_version_2.json
 cce73e0 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
11aac72 
  ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 6981205 

Diff: https://reviews.apache.org/r/47722/diff/


Testing
---

Tested Ranger installation on centos 6


Thanks,

Mugdha Varadkar

Re: Review Request 47634: AMBARI-16788: Add HDFS keytab on RegionServer node

2016-05-23 Thread Gautam Borad


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47634/
---

(Updated May 23, 2016, 10:09 a.m.)


Review request for Ambari, Alejandro Fernandez, Robert Levas, Sumit Mohanty, 
and Velmurugan Periasamy.


Changes
---

Sumit, thanks for the suggestion. This new review request contains logic of 
creating the HDFS dir only from HBASE Master node, thus preventing the HDFS 
headless keytab to be required on the RegionServer node.


Bugs: AMBARI-16788
https://issues.apache.org/jira/browse/AMBARI-16788


Repository: ambari


Description
---

Currently HDFS keytab is not distributed to HBASE RegionServer node. In such 
case Region Server restart will fail, since it wont find the keytab.
The keytab is required to create HDFS dir to store audit logs to.

Added the keytab in kerberos.json file.


Diffs (updated)
-

  
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/hbase_master.py
 840411f 
  
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/hbase_regionserver.py
 4001912 
  
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/setup_ranger_hbase.py
 abb47e5 

Diff: https://reviews.apache.org/r/47634/diff/


Testing
---

Tested on 3 node cluster and ensured that the keytab is available on 
RegionServer node.


Thanks,

Gautam Borad

Re: Review Request 47642: Add HAWQ Standby wizard should recommend HAWQSTANDBY correctly

2016-05-23 Thread Alexander Denissov


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47642/#review134328
---




ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py 
(line 52)


what is standby was also already assigned and is at position 0 in 
hawqMasterHosts[] ? The the logic will break apart, or the whole method will 
not even be called if standby is assigned ?



ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py 
(line 55)


why just not say availableHosts[0] as it is the same as availableHosts[:1] 
but less awkward ?


- Alexander Denissov


On May 21, 2016, 12:07 a.m., Matt wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47642/
> ---
> 
> (Updated May 21, 2016, 12:07 a.m.)
> 
> 
> Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, and Lav 
> Jain.
> 
> 
> Bugs: AMBARI-16790
> https://issues.apache.org/jira/browse/AMBARI-16790
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> On assign masters page of Add HAWQ Standby wizard, a best effort should be 
> made such that HAWQSTANDBY is not co-located with HAWQMASTER or AMBARI_SERVER.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py
>  0590604 
>   ambari-server/src/test/python/stacks/2.3/HAWQ/test_service_advisor.py 
> 905d455 
> 
> Diff: https://reviews.apache.org/r/47642/diff/
> 
> 
> Testing
> ---
> 
> Manually tested.
> 
> Added unit tests:
> ```
> test_createComponentLayoutRecommendations_hawq_1_Host 
> (test_stack_advisor.TestHDP23StackAdvisor) ... ok
> test_createComponentLayoutRecommendations_hawq_3_Hosts 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test that HAWQSTANDBY is recommended on a 3-node cluster ... ok
> test_createComponentLayoutRecommendations_hawqsegment_add_service_wizard_already_installed
>  (test_stack_advisor.TestHDP23StackAdvisor)
> Test that HAWQSEGMENT does not get recommended during Add Service Wizard, 
> when HAWQ has already been installed ... ok
> test_createComponentLayoutRecommendations_hawqsegment_add_service_wizard_to_be_installed
>  (test_stack_advisor.TestHDP23StackAdvisor)
> Test that HAWQSEGMENT gets recommended correctly during Add Service Wizard, 
> when HAWQ is selected for installation ... ok
> test_createComponentLayoutRecommendations_hawqsegment_cluster_install 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test that HAWQSEGMENT gets recommended correctly during Cluster Install 
> Wizard, when HAWQ is selected for installation ... ok
> test_createComponentLayoutRecommendations_no_hawq_3_Hosts 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test no failures when there are no HAWQ components ... ok
> test_createComponentLayoutRecommendations_pxf_add_service_wizard_already_installed
>  (test_stack_advisor.TestHDP23StackAdvisor)
> Test that PXF does not get recommended during Add Service Wizard, when PXF 
> has already been installed ... ok
> test_createComponentLayoutRecommendations_pxf_add_service_wizard_to_be_installed
>  (test_stack_advisor.TestHDP23StackAdvisor)
> Test that PXF gets recommended correctly during Add Service Wizard, when PXF 
> is selected for installation ... ok
> test_createComponentLayoutRecommendations_pxf_cluster_install 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test that PXF gets recommended correctly during Cluster Install Wizard, when 
> PXF is selected for installation ... ok
> test_getComponentLayoutValidations_hawq_3_Hosts 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test layout validations for HAWQ components on a 3-node cluster ... ok
> test_getComponentLayoutValidations_hawqsegment_not_co_located_with_datanode 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test validation warning for HAWQ segment not colocated with DATANODE ... ok
> test_getComponentLayoutValidations_nohawq_3_Hosts 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test no failures when there are no HAWQ components on a 3-node cluster ... ok
> test_getComponentLayoutValidations_pxf_co_located_with_nn_and_dn 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test NO warning is generated when PXF is co-located with NAMENODE and 
> DATANODE ... ok
> test_getComponentLayoutValidations_pxf_not_co_located_with_dn 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test warning is generated when PXF is not co-located with NAMENODE or 
> DATANODE ... ok
> test_getComponentLayoutValidations_pxf_not_co_located_with_nn 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test warning is generated when PXF is not co-located with NAMENODE ... ok
>

Re: Review Request 47018: [preview] "ambari-server upgrade" shouldn't automatically add stack configs

2016-05-23 Thread Dmitro Lisnichenko



> On May 19, 2016, 5:20 p.m., Nate Cole wrote:
> > ambari-server/src/main/resources/property-schema.xsd, lines 41-42
> > 
> >
> > Requiring both?  Both should be optional I would think so that we can 
> > add exceptions, not the-rule.

As we discussed in jira, we are going to have both tags mandatory in order to 
enforce devs to put proper upgrade policies when adding a new property.


> On May 19, 2016, 5:20 p.m., Nate Cole wrote:
> > ambari-server/src/main/resources/property-schema.xsd, lines 68-70
> > 
> >
> > All 3 required?

Yes, these are mandatory attributes for both on-ambari-upgrade and 
on-stack-upgrade tags


> On May 19, 2016, 5:20 p.m., Nate Cole wrote:
> > script.sh, lines 1-5
> > 
> >
> > What is this?

that was a draft script that batch modifies all xml files with properties to 
set up upgrade policies. Will not be present at a final patch


- Dmitro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47018/#review133941
---


On May 18, 2016, 8:11 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47018/
> ---
> 
> (Updated May 18, 2016, 8:11 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, 
> and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-16272
> https://issues.apache.org/jira/browse/AMBARI-16272
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Today, "ambari-server upgrade" will automatically add stack configs.
> However, it also causes problems when default properties or properties with 
> default value such as "localhost" end up being added.
> 
> This led to many bugs. E.g., cluster with NameNode HA shouldn't automatically 
> add dfs.namenode.secondary.http-address
> 
> This logic today will even add new config types. E.g., add ranger-env even 
> though Ranger is not installed. If the customer then upgrades the stack from 
> HDP 2.2 to 2.3, and then adds Ranger, they can get the wrong configs.
> If we change this behavior, it's good to do so in a major release such as 2.4
> 
> We add required xml tags/attributes to properties:
> 
>   prop_name
>   prop_val
>   
>   
> 
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java
>  fb3ae69 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackManager.java 
> 8a352bd 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 
> 854dd0b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/PropertyInfo.java 
> 34b3ba1 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/PropertyUpgradeBehavior.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java
>  2e857ed 
>   ambari-server/src/main/resources/property-schema.xsd PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderHDP22Test.java
>  a4a3108 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderTest.java
>  8f53f6a 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/PropertyInfoTest.java
>  b11c5d8 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/ServicePropertiesTest.java
>  PRE-CREATION 
>   ambari-server/src/test/resources/propertiesFile.xsd PRE-CREATION 
>   script.sh PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/47018/diff/
> 
> 
> Testing
> ---
> 
> patch preview for now. Feedback is welcome
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Re: Review Request 47643: Capacity Scheduler View - Edit and validate queue name

2016-05-23 Thread Sreenath Somarajapuram


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47643/#review134326
---


Ship it!




Ship It!

- Sreenath Somarajapuram


On May 23, 2016, 5:25 a.m., Akhil PB wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47643/
> ---
> 
> (Updated May 23, 2016, 5:25 a.m.)
> 
> 
> Review request for Ambari and Sreenath Somarajapuram.
> 
> 
> Bugs: AMBARI-16792
> https://issues.apache.org/jira/browse/AMBARI-16792
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Capacity Scheduler View - Edit and validate queue name
> 
> 
> Diffs
> -
> 
>   
> contrib/views/capacity-scheduler/src/main/resources/ui/app/components/queueMapping.js
>  02062a6 
>   
> contrib/views/capacity-scheduler/src/main/resources/ui/app/controllers/editqueue.js
>  69be18c 
>   
> contrib/views/capacity-scheduler/src/main/resources/ui/app/controllers/queuesconf.js
>  89d0e89 
>   
> contrib/views/capacity-scheduler/src/main/resources/ui/app/styles/application.less
>  a60d39d 
>   
> contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/capsched/queuesconf.hbs
>  d57b2ba 
>   
> contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/capsched/queuesconf/editqueue.hbs
>  c5c910a 
> 
> Diff: https://reviews.apache.org/r/47643/diff/
> 
> 
> Testing
> ---
> 
> Manual Unit Testing
> 
> 
> Thanks,
> 
> Akhil PB
> 
>

Re: Review Request 47627: hawq_standby_address_host property should be removed from configuration if HAWQSTANDBY component does not exist in BP

2016-05-23 Thread Alexander Denissov


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47627/#review134325
---




ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
 (line 3086)


Since this method is called for every property, we should optimize the 
performance by reaching the decision faster in 99% of cases where the 
conditions do not apply.

If setOfHawqPropertyNamesNonHA.contains(propertyName) operation is less 
expensive (O(1) for the set) then  
topology.getHostGroupsForComponent(HAWQSTANDBY).size() (which might search for 
component inside each host group), then maybe contains() operation should be 
the first check in the if condition.

Also, merging 2 ifs into 1 with && will be more readable.


- Alexander Denissov


On May 20, 2016, 10:11 p.m., bhuvnesh chaudhary wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47627/
> ---
> 
> (Updated May 20, 2016, 10:11 p.m.)
> 
> 
> Review request for Ambari, Alexander Denissov, Matt, and Robert Nettleton.
> 
> 
> Bugs: AMBARI-16786
> https://issues.apache.org/jira/browse/AMBARI-16786
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> hawq_standby_address_host property should be removed from configuration if 
> HAWQSTANDBY component does not exist in BP
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
>  1ddc4e1 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
>  5c6aeaf 
> 
> Diff: https://reviews.apache.org/r/47627/diff/
> 
> 
> Testing
> ---
> 
> yes. test case added.
> 
> 
> Thanks,
> 
> bhuvnesh chaudhary
> 
>

Re: Review Request 47701: AMBARI-16642: After enabling HTTPS for YARN, NodeManager JVM Metrics on YARN Heatmaps show NA.

2016-05-23 Thread Dmytro Sen


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47701/#review134322
---


Ship it!




Ship It!

- Dmytro Sen


On Май 23, 2016, 5:59 д.п., Qin Liu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47701/
> ---
> 
> (Updated Май 23, 2016, 5:59 д.п.)
> 
> 
> Review request for Ambari, Dmytro Sen and Sid Wagle.
> 
> 
> Bugs: AMBARI-16642
> https://issues.apache.org/jira/browse/AMBARI-16642
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> After enabling HTTPS for YARN, "NodeManager GC Time" and "NodeManager JVM 
> Heap Memory Used" widgets on YARN Heatmaps show NA. The root cause is that 
> yarn.nodemanager.webapp.address rather than 
> yarn.nodemanager.webapp.https.address is used to get JMX metrics. 
> 
> Steps to reproduce:
> 1. install a cluster with default from Ambari Web UI.
> 2. configuring SSL for HDFS, YARN, and MapReduce.
> 3. enable HTTPS for YARN
> 1). Set the following property in Advanced yarn-site from Ambari Web UI:
> yarn.http.policy=HTTPS_ONLY
> 2). Add the following property in Custom yarn-site from Ambari Web UI:
> yarn.nodemanager.webapp.https.address=0.0.0.0:8044
> 4. "NodeManager GC Time" and "NodeManager JVM Heap Memory Used" widgets on 
> YARN Heatmaps will show NA.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractProviderModule.java
>  36be39e 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/JMXHostProviderTest.java
>  a6e5501 
> 
> Diff: https://reviews.apache.org/r/47701/diff/
> 
> 
> Testing
> ---
> 
> 1. The fix has been tested with HDP UI
> "NodeManager GC Time" and "NodeManager JVM Heap Memory Used" metrics show up 
> after apply the fix.
> 
> 2. JMXHostProviderTest.java has been modified to verify JMX protocol and port 
> for NodeManager.
> Test set: org.apache.ambari.server.controller.internal.JMXHostProviderTest
> ---
> Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 120.274 sec - 
> in org.apache.ambari.server.controller.internal.JMXHostProviderTest
> 
> 3. The latest result of running ambari-server tests:
> Results :
> Failed tests: 
>   DataStoreImplTest.testFind:409 
>   Unexpected method call EntityManager.getDelegate():
> EntityManager.find(capture(Nothing captured yet), 99): expected: 1, 
> actual: 0
> EntityManager.close(): expected: 1, actual: 0
>   DataStoreImplTest.testFindAll:453 
>   Unexpected method call EntityManager.getDelegate():
> EntityManager.createQuery("SELECT e FROM 
> DS_DataStoreImplTest$TestEntity_1 e WHERE e.DS_id=99"): expected: 1, actual: 0
> EntityManager.close(): expected: 1, actual: 0
>   DataStoreImplTest.testFindAll_multiple:518 
>   Unexpected method call EntityManager.getDelegate():
> EntityManager.createQuery("SELECT e FROM 
> DS_DataStoreImplTest$TestEntity_1 e WHERE e.DS_name='foo'"): expected: 1, 
> actual: 0
> EntityManager.close(): expected: 1, actual: 0
>   DataStoreImplTest.testRemove:370 
>   Unexpected method call EntityManager.getDelegate():
> EntityManager.getReference(capture(Nothing captured yet), 99): expected: 
> 1, actual: 0
> EntityManager.remove(EasyMock for interface 
> org.eclipse.persistence.dynamic.DynamicEntity): expected: 1, actual: 0
> EntityManager.close(): expected: 1, actual: 0
>   DataStoreImplTest.testStore_create:128 
>   Unexpected method call EntityManager.getDelegate():
> EntityManager.find(capture(Nothing captured yet), "bar"): expected: 1, 
> actual: 0
> EntityManager.find(capture(Nothing captured yet), 99): expected: 1, 
> actual: 0
> EntityManager.persist(capture(Nothing captured yet)): expected: 1, 
> actual: 0
> EntityManager.persist(capture(Nothing captured yet)): expected: 1, 
> actual: 0
> EntityManager.close(): expected: 1, actual: 0
>   DataStoreImplTest.testStore_create_largeEntity:222 
>   Unexpected method call EntityManager.getDelegate():
> EntityManager.close(): expected: 1, actual: 0
>   DataStoreImplTest.testStore_create_longStringValue:182 
>   Unexpected method call EntityManager.getDelegate():
> EntityManager.find(capture(Nothing captured yet), 99): expected: 1, 
> actual: 0
> EntityManager.close(): expected: 1, actual: 0
>   DataStoreImplTest.testStore_update:276 
>   Unexpected method call EntityManager.getDelegate():
> EntityManager.find(capture(Nothing captured yet), "bar"): expected: 1, 
> actual: 0
> EntityManager.find(capture(Nothing captured yet), 99): expected: 1, 
> actual: 0
> EntityManager.persist(capture(Nothing captured yet)): expected: 1, 
> actual: 0
>

Re: Review Request 47694: Hive View and Pig View : one user overriding job details of other user in database

2016-05-23 Thread DIPAYAN BHOWMICK


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47694/#review134317
---


Ship it!




Ship It!

- DIPAYAN BHOWMICK


On May 22, 2016, 9:35 a.m., Nitiraj Rathore wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47694/
> ---
> 
> (Updated May 22, 2016, 9:35 a.m.)
> 
> 
> Review request for Ambari, DIPAYAN BHOWMICK, Gaurav Nagar, Pallav 
> Kulshreshtha, and Ashwin Rajeev.
> 
> 
> Bugs: AMBARI-16242
> https://issues.apache.org/jira/browse/AMBARI-16242
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> corrected broken test cases.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
>  eb0a3aa 
>   
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog240Test.java
>  90165a8 
>   
> ambari-server/src/test/java/org/apache/ambari/server/view/persistence/DataStoreImplTest.java
>  d14f120 
> 
> Diff: https://reviews.apache.org/r/47694/diff/
> 
> 
> Testing
> ---
> 
> added and modified test cases.
> 
> 
> Thanks,
> 
> Nitiraj Rathore
> 
>

Review Request 47715: Ambari Agent security bypassed in Python=>2.7.9

2016-05-23 Thread Andrew Onischuk


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47715/
---

Review request for Ambari and Dmitro Lisnichenko.


Bugs: AMBARI-16810
https://issues.apache.org/jira/browse/AMBARI-16810


Repository: ambari


Description
---

We hard-coded the Ambari Agents to ignore certification
verification. But the reason why this was required was Python be un-secure by
default:  
  


That method will cause signed certificates to not serve any purpose & is
discouraged by RedHat & Python security experts:

> "It is also possible, though highly discouraged , to globally disable
verification by monkeypatching the ssl module in versions of Python"

Instead we should abstract it to a setting (e.g. ssl_verify_cert) in the
ambari-agent.ini such that users can turn certification verification if they
provide a signed/trusted certificate.


Diffs
-

  ambari-agent/conf/unix/ambari-agent.ini 4ec16d6 
  ambari-agent/src/main/python/ambari_agent/AmbariConfig.py f849fd1 
  ambari-agent/src/main/python/ambari_agent/Controller.py aee0eec 
  ambari-agent/src/main/python/ambari_agent/NetUtil.py 1d5cb29 
  ambari-agent/src/main/python/ambari_agent/main.py 5340239 

Diff: https://reviews.apache.org/r/47715/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Re: Review Request 47701: AMBARI-16642: After enabling HTTPS for YARN, NodeManager JVM Metrics on YARN Heatmaps show NA.

2016-05-23 Thread Qin Liu


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47701/
---

(Updated May 23, 2016, 5:59 a.m.)


Review request for Ambari, Dmytro Sen and Sid Wagle.


Summary (updated)
-

AMBARI-16642: After enabling HTTPS for YARN, NodeManager JVM Metrics on YARN 
Heatmaps show NA.


Bugs: AMBARI-16642
https://issues.apache.org/jira/browse/AMBARI-16642


Repository: ambari


Description
---

After enabling HTTPS for YARN, "NodeManager GC Time" and "NodeManager JVM Heap 
Memory Used" widgets on YARN Heatmaps show NA. The root cause is that 
yarn.nodemanager.webapp.address rather than 
yarn.nodemanager.webapp.https.address is used to get JMX metrics. 

Steps to reproduce:
1. install a cluster with default from Ambari Web UI.
2. configuring SSL for HDFS, YARN, and MapReduce.
3. enable HTTPS for YARN
1). Set the following property in Advanced yarn-site from Ambari Web UI:
yarn.http.policy=HTTPS_ONLY
2). Add the following property in Custom yarn-site from Ambari Web UI:
yarn.nodemanager.webapp.https.address=0.0.0.0:8044
4. "NodeManager GC Time" and "NodeManager JVM Heap Memory Used" widgets on YARN 
Heatmaps will show NA.


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractProviderModule.java
 36be39e 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/JMXHostProviderTest.java
 a6e5501 

Diff: https://reviews.apache.org/r/47701/diff/


Testing (updated)
---

1. The fix has been tested with HDP UI
"NodeManager GC Time" and "NodeManager JVM Heap Memory Used" metrics show up 
after apply the fix.

2. JMXHostProviderTest.java has been modified to verify JMX protocol and port 
for NodeManager.
Test set: org.apache.ambari.server.controller.internal.JMXHostProviderTest
---
Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 120.274 sec - 
in org.apache.ambari.server.controller.internal.JMXHostProviderTest

3. The latest result of running ambari-server tests:
Results :
Failed tests: 
  DataStoreImplTest.testFind:409 
  Unexpected method call EntityManager.getDelegate():
EntityManager.find(capture(Nothing captured yet), 99): expected: 1, actual: 0
EntityManager.close(): expected: 1, actual: 0
  DataStoreImplTest.testFindAll:453 
  Unexpected method call EntityManager.getDelegate():
EntityManager.createQuery("SELECT e FROM DS_DataStoreImplTest$TestEntity_1 
e WHERE e.DS_id=99"): expected: 1, actual: 0
EntityManager.close(): expected: 1, actual: 0
  DataStoreImplTest.testFindAll_multiple:518 
  Unexpected method call EntityManager.getDelegate():
EntityManager.createQuery("SELECT e FROM DS_DataStoreImplTest$TestEntity_1 
e WHERE e.DS_name='foo'"): expected: 1, actual: 0
EntityManager.close(): expected: 1, actual: 0
  DataStoreImplTest.testRemove:370 
  Unexpected method call EntityManager.getDelegate():
EntityManager.getReference(capture(Nothing captured yet), 99): expected: 1, 
actual: 0
EntityManager.remove(EasyMock for interface 
org.eclipse.persistence.dynamic.DynamicEntity): expected: 1, actual: 0
EntityManager.close(): expected: 1, actual: 0
  DataStoreImplTest.testStore_create:128 
  Unexpected method call EntityManager.getDelegate():
EntityManager.find(capture(Nothing captured yet), "bar"): expected: 1, 
actual: 0
EntityManager.find(capture(Nothing captured yet), 99): expected: 1, actual: 0
EntityManager.persist(capture(Nothing captured yet)): expected: 1, actual: 0
EntityManager.persist(capture(Nothing captured yet)): expected: 1, actual: 0
EntityManager.close(): expected: 1, actual: 0
  DataStoreImplTest.testStore_create_largeEntity:222 
  Unexpected method call EntityManager.getDelegate():
EntityManager.close(): expected: 1, actual: 0
  DataStoreImplTest.testStore_create_longStringValue:182 
  Unexpected method call EntityManager.getDelegate():
EntityManager.find(capture(Nothing captured yet), 99): expected: 1, actual: 0
EntityManager.close(): expected: 1, actual: 0
  DataStoreImplTest.testStore_update:276 
  Unexpected method call EntityManager.getDelegate():
EntityManager.find(capture(Nothing captured yet), "bar"): expected: 1, 
actual: 0
EntityManager.find(capture(Nothing captured yet), 99): expected: 1, actual: 0
EntityManager.persist(capture(Nothing captured yet)): expected: 1, actual: 0
EntityManager.close(): expected: 1, actual: 0
  DataStoreImplTest.testStore_update_longStringValue:328 
  Unexpected method call EntityManager.getDelegate():
EntityManager.find(capture(Nothing captured yet), 99): expected: 1, actual: 0
EntityManager.close(): expected: 1, actual: 0
Tests in error: 
  PersistKeyValueImplTest.testMultiThreaded:103 » OutOfMemory unable to create 
n...

65 matches

Mail list logo