date:20171208

Re: Review Request 64397: AMBARI-22602. Add 'clusterSettings' and 'stackSettings' parameters in Execution Command.

2017-12-08 Thread Madhuvanthi Radhakrishnan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64397/#review193284
---


Fix it, then Ship it!





ambari-server/src/main/java/org/apache/ambari/server/resources/RootLevelSettingsManager.java
Line 57 (original)


Are we using this map anywhere?


- Madhuvanthi Radhakrishnan


On Dec. 7, 2017, 8:11 a.m., Swapan Shridhar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64397/
> ---
> 
> (Updated Dec. 7, 2017, 8:11 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya and Madhuvanthi Radhakrishnan.
> 
> 
> Bugs: AMBARI-22602
> https://issues.apache.org/jira/browse/AMBARI-22602
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> AMBARI-22198 added "stack settings", and AMBARI-22196 introduced "cluster 
> settings" in Ambari.
> 
> This review adds 2 new parameters to Execution Command : **(1).** 
> clusterSettings and **(2).** stackSettings, and adds their respective set of 
> key-values pairs.
> 
> - This enables these parameters to be passed in to the ambari-agent's 
> **command*.json**.
> 
> 
> Test cases are tracked in : AMBARI-22603
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/ExecutionCommand.java
>  5ee4bf6 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java
>  e7dea06 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  59e6622 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClientConfigResourceProvider.java
>  9043297 
>   
> ambari-server/src/main/java/org/apache/ambari/server/resources/RootLevelSettingsManager.java
>  8bd24f2 
> 
> 
> Diff: https://reviews.apache.org/r/64397/diff/1/
> 
> 
> Testing
> ---
> 
> Tested on Live cluster doing following : 
> (1). Start Service 
> (2). Restart Service 
> (3). Stop Service 
> (4). Refreshing client configs
> (5). Adding Client.
> 
> 
> **Snippet from command*.json as part of testing:**
> {
> ...
> ...
> "clusterSettings": {
> "security_enabled": "false",
> "hide_yarn_memory_widget": "false",
> "enable_external_ranger": "false",
> "override_uid": "true",
> "kerberos_domain": "EXAMPLE.COM",
> "one_dir_per_partition": "false",
> "repo_ubuntu_template": "{{package_type}} {{base_url}} 
> {{components}}",
> "ignore_groupsusers_create": "false",
> "alerts_repeat_tolerance": "1",
> "namenode_rolling_restart_timeout": "4200",
> "fetch_nonlocal_groups": "true",
> "manage_dirs_on_root": "true",
> "recovery_lifetime_max_count": "1024",
> "agent_mounts_ignore_list": "",
> "ignore_bad_mounts": "false",
> "recovery_window_in_minutes": "60",
> "sysprep_skip_copy_tarballs_hdfs": "false",
> "recovery_type": "AUTO_START",
> "user_group": "hadoop",
> "namenode_rolling_restart_safemode_exit_timeout": "3600",
> "recovery_retry_interval": "5",
> "sysprep_skip_copy_oozie_share_lib_to_hdfs": "false",
> "sysprep_skip_setup_jce": "false",
> "manage_hive_fsroot": "true",
> "service_check_type": "full",
> "recovery_enabled": "true",
> "recovery_max_count": "6",
> "sysprep_skip_create_users_and_groups": "false",
> "smokeuser_keytab": "/etc/security/keytabs/smokeuser.headless.keytab",
> "managed_hdfs_resource_property_names": "false",
> "smokeuser": "ambari-qa",
> "sysprep_skip_copy_fast_jar_hdfs": "false"
> },
> ...
> ...
> ...
> ...
> ...
> ...
> ...
> "stackSettings": {
> "stack_features": "{\n  \"HDP\": {\n\"stack_features\": [\n  
> {\n\"name\": \"snappy\",\n\"description\": \"Snappy 
> compressor/decompressor support\",\n\"min_version\": \"2.0.0.0\",\n   
>  \"max_version\": \"2.2.0.0\"\n  },\n  {\n\"name\": 
> \"lzo\",\n\"description\": \"LZO libraries support\",\n
> \"min_version\": \"2.2.1.0\"\n  },\n  {\n\"name\": 
> \"express_upgrade\",\n\"description\": \"Express upgrade support\",\n 
>\"min_version\": \"2.1.0.0\"\n  },\n  {\n\"name\": 
> \"rolling_upgrade\",\n\"description\": \"Rolling upgrade support\",\n 
>\"min_version\": \"2.2.0.0\"\n  },\n  {\n\"name\": 
> \"kafka_acl_migration_support\",\n\"description\": \"ACL migration 
> support\",\n\"min_version\": \"2.3.4.0\"\n  },\n  {\n
> \"name\": \"secure_zookeeper\

Re: Review Request 64393: AMBARI-22607 : Design and implement an AD job scheduler.

2017-12-08 Thread Aravindan Vijayan



> On Dec. 7, 2017, 3:03 a.m., Sid Wagle wrote:
> > ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/detection/pointintime/PointInTimeSubsystem.scala
> > Lines 61 (patched)
> > 
> >
> > Better model will be to add a listener and handle events.

Will handle in subsequent refinements.


- Aravindan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64393/#review193087
---


On Dec. 8, 2017, 9:27 p.m., Aravindan Vijayan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64393/
> ---
> 
> (Updated Dec. 8, 2017, 9:27 p.m.)
> 
> 
> Review request for Ambari, Jayush Luniya and Sid Wagle.
> 
> 
> Bugs: AMBARI-22607
> https://issues.apache.org/jira/browse/AMBARI-22607
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Design and implement a scheduler for starting and managing AD jobs on Spark.
> The Spark runtime configuration will be analyzed and refined in follow up 
> JIRAs.
> 
> 
> Diffs
> -
> 
>   
> ambari-metrics/ambari-metrics-anomaly-detection-service/conf/unix/config.yaml 
> 85e4004 
>   ambari-metrics/ambari-metrics-anomaly-detection-service/pom.xml 50d7ef6 
>   
> ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/java/org/apache/ambari/metrics/adservice/prototype/core/MetricSparkConsumer.java
>  addeda7 
>   
> ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/app/AnomalyDetectionApp.scala
>  2d0dbdf 
>   
> ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/app/AnomalyDetectionAppConfig.scala
>  58efa97 
>   
> ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/app/AnomalyDetectionAppModule.scala
>  68e9df9 
>   
> ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/configuration/DetectionServiceConfiguration.scala
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/configuration/SparkConfiguration.scala
>  30efdc7 
>   
> ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/db/PhoenixAnomalyStoreAccessor.scala
>  53e6dee 
>   
> ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/detection/AdJobManager.scala
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/detection/SparkApplicationRunner.scala
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/detection/Subsystem.scala
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/detection/pointintime/EmaSparkDriver.scala
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/detection/pointintime/PointInTimeSubsystem.scala
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/metadata/MetricDefinitionService.scala
>  52ce39e 
>   
> ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/metadata/MetricDefinitionServiceImpl.scala
>  b9b4a7c 
>   
> ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/metadata/MetricSourceDefinition.scala
>  47b1499 
>   
> ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/resource/DetectionResource.scala
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/resource/MetricDefinitionResource.scala
>  442bf46 
>   
> ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/service/DetectionService.scala
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/service/DetectionServiceImpl.scala
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/service/MetricDefinitionService.scala
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/servi

Re: Review Request 64393: AMBARI-22607 : Design and implement an AD job scheduler.

2017-12-08 Thread Aravindan Vijayan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64393/
---

(Updated Dec. 8, 2017, 9:27 p.m.)


Review request for Ambari, Jayush Luniya and Sid Wagle.


Bugs: AMBARI-22607
https://issues.apache.org/jira/browse/AMBARI-22607


Repository: ambari


Description
---

Design and implement a scheduler for starting and managing AD jobs on Spark.
The Spark runtime configuration will be analyzed and refined in follow up JIRAs.


Diffs (updated)
-

  ambari-metrics/ambari-metrics-anomaly-detection-service/conf/unix/config.yaml 
85e4004 
  ambari-metrics/ambari-metrics-anomaly-detection-service/pom.xml 50d7ef6 
  
ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/java/org/apache/ambari/metrics/adservice/prototype/core/MetricSparkConsumer.java
 addeda7 
  
ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/app/AnomalyDetectionApp.scala
 2d0dbdf 
  
ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/app/AnomalyDetectionAppConfig.scala
 58efa97 
  
ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/app/AnomalyDetectionAppModule.scala
 68e9df9 
  
ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/configuration/DetectionServiceConfiguration.scala
 PRE-CREATION 
  
ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/configuration/SparkConfiguration.scala
 30efdc7 
  
ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/db/PhoenixAnomalyStoreAccessor.scala
 53e6dee 
  
ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/detection/AdJobManager.scala
 PRE-CREATION 
  
ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/detection/SparkApplicationRunner.scala
 PRE-CREATION 
  
ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/detection/Subsystem.scala
 PRE-CREATION 
  
ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/detection/pointintime/EmaSparkDriver.scala
 PRE-CREATION 
  
ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/detection/pointintime/PointInTimeSubsystem.scala
 PRE-CREATION 
  
ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/metadata/MetricDefinitionService.scala
 52ce39e 
  
ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/metadata/MetricDefinitionServiceImpl.scala
 b9b4a7c 
  
ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/metadata/MetricSourceDefinition.scala
 47b1499 
  
ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/resource/DetectionResource.scala
 PRE-CREATION 
  
ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/resource/MetricDefinitionResource.scala
 442bf46 
  
ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/service/DetectionService.scala
 PRE-CREATION 
  
ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/service/DetectionServiceImpl.scala
 PRE-CREATION 
  
ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/service/MetricDefinitionService.scala
 PRE-CREATION 
  
ambari-metrics/ambari-metrics-anomaly-detection-service/src/main/scala/org/apache/ambari/metrics/adservice/service/MetricDefinitionServiceImpl.scala
 PRE-CREATION 
  
ambari-metrics/ambari-metrics-anomaly-detection-service/src/test/resources/config.yaml
 6b09499 
  
ambari-metrics/ambari-metrics-anomaly-detection-service/src/test/scala/org/apache/ambari/metrics/adservice/app/AnomalyDetectionAppConfigTest.scala
 76391a0 
  
ambari-metrics/ambari-metrics-anomaly-detection-service/src/test/scala/org/apache/ambari/metrics/adservice/metadata/MetricDefinitionServiceTest.scala
 d3454f2 
  
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-admanager-config.xml
 9862f10 
  
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py
 a18b8f6 


Diff: https://reviews.apache.org/r/64393/diff/2/

Changes: https://reviews.apache.org/r/64393/diff/1-2/


Testing
---

Manually tested.
Current unit tests pass.
New Unit tests being worked on...

Re: Review Request 64397: AMBARI-22602. Add 'clusterSettings' and 'stackSettings' parameters in Execution Command.

2017-12-08 Thread Attila Doroszlai


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64397/#review193274
---


Fix it, then Ship it!





ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java
Lines 436 (patched)


* no need for explicit type arg in Maps.newHashMap()
* empty map should be created lazily in `else` below



ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java
Lines 438 (patched)


According to IDEA there is no need to include the `clusterSettings.values() 
!= null` check, as it is always true if `clusterSettings != null` is true.



ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java
Lines 449-451 (patched)


Please use `collect(toMap())` (as above) instead of `forEach` and `put`



ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
Lines 2676-2695 (patched)


Can you please extract the logic to 2 utility methods to avoid code 
duplication among `AmbariCustomCommandExecutionHelper`, 
`AmbariManagementControllerImpl` and `ClientConfigResourceProvider`?


- Attila Doroszlai


On Dec. 7, 2017, 9:11 a.m., Swapan Shridhar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64397/
> ---
> 
> (Updated Dec. 7, 2017, 9:11 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya and Madhuvanthi Radhakrishnan.
> 
> 
> Bugs: AMBARI-22602
> https://issues.apache.org/jira/browse/AMBARI-22602
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> AMBARI-22198 added "stack settings", and AMBARI-22196 introduced "cluster 
> settings" in Ambari.
> 
> This review adds 2 new parameters to Execution Command : **(1).** 
> clusterSettings and **(2).** stackSettings, and adds their respective set of 
> key-values pairs.
> 
> - This enables these parameters to be passed in to the ambari-agent's 
> **command*.json**.
> 
> 
> Test cases are tracked in : AMBARI-22603
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/ExecutionCommand.java
>  5ee4bf6 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java
>  e7dea06 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  59e6622 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClientConfigResourceProvider.java
>  9043297 
>   
> ambari-server/src/main/java/org/apache/ambari/server/resources/RootLevelSettingsManager.java
>  8bd24f2 
> 
> 
> Diff: https://reviews.apache.org/r/64397/diff/1/
> 
> 
> Testing
> ---
> 
> Tested on Live cluster doing following : 
> (1). Start Service 
> (2). Restart Service 
> (3). Stop Service 
> (4). Refreshing client configs
> (5). Adding Client.
> 
> 
> **Snippet from command*.json as part of testing:**
> {
> ...
> ...
> "clusterSettings": {
> "security_enabled": "false",
> "hide_yarn_memory_widget": "false",
> "enable_external_ranger": "false",
> "override_uid": "true",
> "kerberos_domain": "EXAMPLE.COM",
> "one_dir_per_partition": "false",
> "repo_ubuntu_template": "{{package_type}} {{base_url}} 
> {{components}}",
> "ignore_groupsusers_create": "false",
> "alerts_repeat_tolerance": "1",
> "namenode_rolling_restart_timeout": "4200",
> "fetch_nonlocal_groups": "true",
> "manage_dirs_on_root": "true",
> "recovery_lifetime_max_count": "1024",
> "agent_mounts_ignore_list": "",
> "ignore_bad_mounts": "false",
> "recovery_window_in_minutes": "60",
> "sysprep_skip_copy_tarballs_hdfs": "false",
> "recovery_type": "AUTO_START",
> "user_group": "hadoop",
> "namenode_rolling_restart_safemode_exit_timeout": "3600",
> "recovery_retry_interval": "5",
> "sysprep_skip_copy_oozie_share_lib_to_hdfs": "false",
> "sysprep_skip_setup_jce": "false",
> "manage_hive_fsroot": "true",
> "service_check_type": "full",
> "recovery_enabled": "true",
> "recovery_max_count": "6",
> "sysprep_skip_create_users_and_groups": "false",
> "smokeuser_keytab": "/etc/security/keytabs/smokeuser.headless.keytab",
> "managed_hdfs_resource_property_names": "false",
> "smokeuser": "ambari-qa",
> "sysprep_skip_copy_fast_jar_hdfs": "false"
>

Re: Review Request 64420: Fix unit tests in feature branch to make them workable

2017-12-08 Thread Attila Doroszlai


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64420/#review193254
---


Ship it!




Ship It!

- Attila Doroszlai


On Dec. 7, 2017, 5:59 p.m., Vitalyi Brodetskyi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64420/
> ---
> 
> (Updated Dec. 7, 2017, 5:59 p.m.)
> 
> 
> Review request for Ambari, Attila Doroszlai and Jayush Luniya.
> 
> 
> Bugs: AMBARI-22614
> https://issues.apache.org/jira/browse/AMBARI-22614
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Fix unit tests in feature branch to make them workable
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackManager.java 
> fc49a24 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/ServiceFactory.java
>  69eb21d 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
>  0b38b36 
>   
> ambari-server/src/test/java/org/apache/ambari/server/actionmanager/ExecutionCommandWrapperTest.java
>  19f940f 
>   
> ambari-server/src/test/java/org/apache/ambari/server/actionmanager/TestActionSchedulerThreading.java
>  22f23c8 
>   
> ambari-server/src/test/java/org/apache/ambari/server/agent/HeartbeatProcessorTest.java
>  8a6025f 
>   
> ambari-server/src/test/java/org/apache/ambari/server/agent/TestHeartbeatHandler.java
>  50d4733 
>   
> ambari-server/src/test/java/org/apache/ambari/server/agent/TestHeartbeatMonitor.java
>  4f8c423 
>   
> ambari-server/src/test/java/org/apache/ambari/server/api/resources/ClusterResourceDefinitionTest.java
>  fc24d2c 
>   
> ambari-server/src/test/java/org/apache/ambari/server/api/resources/ServiceResourceDefinitionTest.java
>  75319ea 
>   
> ambari-server/src/test/java/org/apache/ambari/server/checks/ComponentExistsInRepoCheckTest.java
>  2fe8ee0 
>   
> ambari-server/src/test/java/org/apache/ambari/server/configuration/RecoveryConfigHelperTest.java
>  040b571 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerImplTest.java
>  21cf518 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerTest.java
>  5c6f374 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/StackDefinedPropertyProviderTest.java
>  aee1952 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderTest.java
>  9e6b98b 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeSummaryResourceProviderTest.java
>  387fa45 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProviderTest.java
>  8e968cc 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/utilities/state/GeneralServiceCalculatedStateTest.java
>  0b6c3f3 
>   
> ambari-server/src/test/java/org/apache/ambari/server/events/listeners/upgrade/HostVersionOutOfSyncListenerTest.java
>  a86d125 
>   ambari-server/src/test/java/org/apache/ambari/server/orm/OrmTestHelper.java 
> a03d7cc 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/ConfigHelperTest.java
>  158c47d 
>   ambari-server/src/test/java/org/apache/ambari/server/state/ServiceTest.java 
> 4e1c139 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/UpgradeHelperTest.java
>  5e5395a 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClusterImplTest.java
>  a52f438 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClusterTest.java
>  4ce7387 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClustersTest.java
>  be4a907 
>   ambari-server/src/test/resources/cluster-settings.xml PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/64420/diff/1/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>

Re: Review Request 64242: AMBARI-22476: Modify bootstrap API to pass Ambari repo URL from UI and retrieve Host OS types

2017-12-08 Thread Sonia Garudi



> On Dec. 4, 2017, 4:20 p.m., Nate Cole wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/bootstrap/SshHostInfo.java
> > Lines 130-132 (patched)
> > 
> >
> > How many URLs are we talking about here.  If more than one, should this 
> > bet a Set?
> 
> Sonia Garudi wrote:
> ambariRepoUrls is a json string having ambari repo URLs for each os type 
> which is sent from the UI.
> 
> Nate Cole wrote:
> Can I see an example?
> 
> Sonia Garudi wrote:
> Example : 
> 
> "[{"os_type":"os1","ambari_repo":"http://ambari-repo1","hosts":["host1","host2"],"ambariRepoUIError":"","hasError":false},{"os_type":"os2","ambari_repo":"http://ambari-repo2","hosts":["host3"],"ambariRepoUIError":"","hasError":false}]";
> The above json string consists of list of 2 objects; each containing the 
> os_type, ambari repository URL and the list of host names for the 
> corresponding os_type. The 2 other fields(i.e. ambariRepoUIError and 
> hasError) are used for UI validation.
> 
> Nate Cole wrote:
> This seems strange in that we end up with two different ways to "get" the 
> URL.  One is that when the UI does not recognize an OS, we ask for it and 
> (presumably) save it in ambari.properties.  Seems as though when we should 
> always get the value from ambari.properties and fail if it's not there.  
> Also, why are we setting all this information  If we have a command for 
> host1, host2, why would we need to know that os-type-2 goes on host3?

The bootstrap.py file first checks if the os type of a host is not same as 
server os type. If its different it checks for the ambari repo property for 
that os type in the ambari.properties file. 
case 1 : If the property is not found, the bootstrap for that host fails with 
code 44. The UI prompts for the ambari repo URL for os_type of hosts which fail 
with code 44. These hosts are bootstrapped with the ambariRepoUrls json string 
which contains the ambari repo urls. These URLs are then saved in the 
ambari.properties file for the respective os type.
case 2 : If the property is found in the properties file, the installation will 
continue without showing a prompt for that os_type.

The hosts field in each object is used for removal of prompt when a host is 
removed.
Implementation of storing of ambari repo url in properties file in done in 
upcoming patches.


- Sonia


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64242/#review192705
---


On Dec. 1, 2017, 7:47 a.m., Sonia Garudi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64242/
> ---
> 
> (Updated Dec. 1, 2017, 7:47 a.m.)
> 
> 
> Review request for Ambari and Nate Cole.
> 
> 
> Bugs: AMBARI-22476
> https://issues.apache.org/jira/browse/AMBARI-22476
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> The hosts failing with error code 44 are bootstrapped along with the ambari 
> repository URLs. The required java beans are changed to include the ambari 
> repository URLs to be passed to bootstrap.py.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/bootstrap/BSRunner.java 
> c7976ee5bb48209478258625532e3420973b4b8c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/bootstrap/SshHostInfo.java
>  86888f499f15aa23807067ee1af745525b9d4619 
>   ambari-server/src/main/python/bootstrap.py 
> f1c53ce45316843fd38624536eed656b029a230a 
>   
> ambari-server/src/test/java/org/apache/ambari/server/bootstrap/BootStrapTest.java
>  b9da013524fee556c6faa2f0236d2c0c4b032750 
>   ambari-server/src/test/python/TestBootstrap.py 
> bea47f498cbb523fbe24622c63b48918d433c717 
>   ambari-web/app/controllers/wizard/step3_controller.js 
> 5ff2cdc1b4bb9da20565fe387765b4fc6728e096 
>   ambari-web/test/controllers/wizard/step3_test.js 
> be1affe28fdcdb3404ebd3fb3fc6e61315483ed2 
> 
> 
> Diff: https://reviews.apache.org/r/64242/diff/1/
> 
> 
> Testing
> ---
> 
> Tests were successful in ambari-web and ambari-server.
> 
> 
> Thanks,
> 
> Sonia Garudi
> 
>

Re: Review Request 64446: Log Search UI: improve histogram display

2017-12-08 Thread Oleg Nechiporenko


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64446/#review193241
---


Ship it!




Ship It!

- Oleg Nechiporenko


On Dec. 8, 2017, 11:33 a.m., Andrii Babiichuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64446/
> ---
> 
> (Updated Dec. 8, 2017, 11:33 a.m.)
> 
> 
> Review request for Ambari and Oleg Nechiporenko.
> 
> 
> Bugs: AMBARI-22615
> https://issues.apache.org/jira/browse/AMBARI-22615
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> 1. Add tooltip over columns
> 2. Add legend
> 3. Add display of selected time interval length
> 4. Reduce the height of the histogram
> 5. When narrowing in on time in the histogram we need to make the bars wider 
> to fill more of the gap
> 6. Also when zooming in we should have text to tell you the gap
> 7. The amount of space between the y axis and the bars needs to be increased
> 
> 
> Diffs
> -
> 
>   ambari-logsearch/ambari-logsearch-web/src/app/classes/histogram-options.ts 
> dee5d98 
>   ambari-logsearch/ambari-logsearch-web/src/app/components/mixins.less 
> a6e5616 
>   
> ambari-logsearch/ambari-logsearch-web/src/app/components/time-histogram/time-histogram.component.html
>  299e46e 
>   
> ambari-logsearch/ambari-logsearch-web/src/app/components/time-histogram/time-histogram.component.less
>  e8d3240 
>   
> ambari-logsearch/ambari-logsearch-web/src/app/components/time-histogram/time-histogram.component.spec.ts
>  9e056be 
>   
> ambari-logsearch/ambari-logsearch-web/src/app/components/time-histogram/time-histogram.component.ts
>  e255166 
>   
> ambari-logsearch/ambari-logsearch-web/src/app/services/logs-container.service.ts
>  e754aa4 
>   ambari-logsearch/ambari-logsearch-web/src/assets/i18n/en.json 6c916aa 
> 
> 
> Diff: https://reviews.apache.org/r/64446/diff/1/
> 
> 
> Testing
> ---
> 
> Executed 221 of 221 SUCCESS (5.452 secs / 5.358 secs)
> 
> 
> Thanks,
> 
> Andrii Babiichuk
> 
>

Review Request 64446: Log Search UI: improve histogram display

2017-12-08 Thread Andrii Babiichuk

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64446/
---

Review request for Ambari and Oleg Nechiporenko.

Bugs: AMBARI-22615
https://issues.apache.org/jira/browse/AMBARI-22615

Repository: ambari

Description
---

1. Add tooltip over columns
2. Add legend
3. Add display of selected time interval length
4. Reduce the height of the histogram
5. When narrowing in on time in the histogram we need to make the bars wider to
fill more of the gap
6. Also when zooming in we should have text to tell you the gap
7. The amount of space between the y axis and the bars needs to be increased

Diffs
-

ambari-logsearch/ambari-logsearch-web/src/app/classes/histogram-options.ts
dee5d98
ambari-logsearch/ambari-logsearch-web/src/app/components/mixins.less a6e5616

ambari-logsearch/ambari-logsearch-web/src/app/components/time-histogram/time-histogram.component.html
299e46e

ambari-logsearch/ambari-logsearch-web/src/app/components/time-histogram/time-histogram.component.less
e8d3240

ambari-logsearch/ambari-logsearch-web/src/app/components/time-histogram/time-histogram.component.spec.ts
9e056be

ambari-logsearch/ambari-logsearch-web/src/app/components/time-histogram/time-histogram.component.ts
e255166

ambari-logsearch/ambari-logsearch-web/src/app/services/logs-container.service.ts
e754aa4
ambari-logsearch/ambari-logsearch-web/src/assets/i18n/en.json 6c916aa

Diff: https://reviews.apache.org/r/64446/diff/1/

Testing
---

Executed 221 of 221 SUCCESS (5.452 secs / 5.358 secs)

Thanks,

Andrii Babiichuk

Re: Review Request 64407: AMBARI-22592: Set ambari.repo. in ambari-server setup script

2017-12-08 Thread Sneha Kanekar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64407/
---

(Updated Dec. 8, 2017, 10:57 a.m.)


Review request for Ambari and Nate Cole.


Changes
---

Removed odd spaces and also corrected the patch according to the latest changes 
in feature branch.


Bugs: AMBARI-22592
https://issues.apache.org/jira/browse/AMBARI-22592


Repository: ambari


Description
---

The ambari-server setup script is updated to allow the user to store/update 
Ambari repo base URL in ambari.properties file for each OS type through command 
line. The command to set Ambari repo URL : ambari-server setup --ambari-repo 



Diffs (updated)
-

  ambari-server/src/main/python/ambari-server.py 188910e 
  ambari-server/src/main/python/ambari_server/serverConfiguration.py 27b4472 
  ambari-server/src/main/python/ambari_server/serverSetup.py 3b41fff 
  ambari-server/src/test/python/TestAmbariServer.py ad4e371 


Diff: https://reviews.apache.org/r/64407/diff/2/

Changes: https://reviews.apache.org/r/64407/diff/1-2/


Testing
---

Tests were successful in ambari-server.


Thanks,

Sneha Kanekar

Re: Review Request 64419: Hive Queries Failing with Missing SnappyCodec Compression Type Due to Hive's Use of MapReduce Admin Env Property

2017-12-08 Thread Dmitro Lisnichenko


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64419/#review193236
---


Ship it!




Ship It!

- Dmitro Lisnichenko


On Dec. 7, 2017, 10:53 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64419/
> ---
> 
> (Updated Dec. 7, 2017, 10:53 p.m.)
> 
> 
> Review request for Ambari, Dmitro Lisnichenko and Nate Cole.
> 
> 
> Bugs: AMBARI-22613
> https://issues.apache.org/jira/browse/AMBARI-22613
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Many queries, similar to the below {{InsertOverwrite}}, are failing after 
> performing an HDP->HDP upgrade. The cause is that Hive does not localize the 
> Tez TAR file, but instead uses the MapReduce2 property 
> {{mapreduce.admin.user.env}}
> 
> ```
> >>>  analyze table studenttab10k compute statistics for columns age ;
> INFO  : Session is already open
> INFO  : Dag name: analyze table studenttab10k compute st...age(Stage-0)
> INFO  : Status: Running (Executing on YARN cluster with App id 
> application_1512568339263_0003)
> 
> [2K
> [2K[36;1mVERTICES  STATUS  TOTAL  COMPLETED  RUNNING  PENDING  
> FAILED  KILLED
> [22;0m[2K
> [2KMap 1RUNNING  1  010   2  
>  0
> Reducer 2 INITED  1  001   0  
>  0
> [2K
> [2K[31;1mVERTICES: 00/02  [>>--] 0%ELAPSED 
> TIME: 9.38 s 
> [22;0m[2K
> [8A[2K
> [2K[36;1mVERTICES  STATUS  TOTAL  COMPLETED  RUNNING  PENDING  
> FAILED  KILLED
> [22;0m[2K
> [2KMap 1RUNNING  1  001   4  
>  0
> Reducer 2 INITED  1  001   0  
>  0
> [2K
> [2K[31;1mVERTICES: 00/02  [>>--] 0%ELAPSED 
> TIME: 14.37 s
> [22;0m[2K
> ERROR : Status: Failed
> ERROR : Vertex failed, vertexName=Map 1, 
> vertexId=vertex_1512568339263_0003_2_00, diagnostics=[Task failed, 
> taskId=task_1512568339263_0003_2_00_00, diagnostics=[TaskAttempt 0 
> failed, info=[Error: Failure while running task:java.lang.RuntimeException: 
> java.io.IOException: Unable to get CompressorType for codec 
> (org.apache.hadoop.io.compress.SnappyCodec). This is most likely due to 
> missing native libraries for the codec.
> Caused by: java.lang.RuntimeException: native snappy library not available: 
> this version of libhadoop was built without snappy support.
>   at 
> org.apache.hadoop.io.compress.SnappyCodec.checkNativeCodeLoaded(SnappyCodec.java:65)
>   at 
> org.apache.hadoop.io.compress.SnappyCodec.getCompressorType(SnappyCodec.java:134)
>   at 
> org.apache.tez.runtime.library.common.sort.impl.ExternalSorter.(ExternalSorter.java:208)
>   ... 18 more
> ], TaskAttempt 1 failed, info=[Error: Failure while running 
> task:java.lang.RuntimeException: java.io.IOException: Unable to get 
> CompressorType for codec (org.apache.hadoop.io.compress.SnappyCodec). This is 
> most likely due to missing native libraries for the codec.
> ```
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py
>  18e297865e 
>   ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/config-upgrade.xml 
> 6e1c81968d 
> 
> 
> Diff: https://reviews.apache.org/r/64419/diff/2/
> 
> 
> Testing
> ---
> 
> Manual upgrade testing.
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Re: Review Request 64397: AMBARI-22602. Add 'clusterSettings' and 'stackSettings' parameters in Execution Command.

Re: Review Request 64393: AMBARI-22607 : Design and implement an AD job scheduler.

Re: Review Request 64393: AMBARI-22607 : Design and implement an AD job scheduler.

Re: Review Request 64397: AMBARI-22602. Add 'clusterSettings' and 'stackSettings' parameters in Execution Command.

Re: Review Request 64420: Fix unit tests in feature branch to make them workable

Re: Review Request 64242: AMBARI-22476: Modify bootstrap API to pass Ambari repo URL from UI and retrieve Host OS types

Re: Review Request 64446: Log Search UI: improve histogram display

Review Request 64446: Log Search UI: improve histogram display

Re: Review Request 64407: AMBARI-22592: Set ambari.repo. in ambari-server setup script

Re: Review Request 64419: Hive Queries Failing with Missing SnappyCodec Compression Type Due to Hive's Use of MapReduce Admin Env Property

10 matches

Site Navigation

Mail list logo

Footer information