Re: Review Request 48065: AMBARI-16949 Metrics Collector API shows NPE if we use wildcard (%25 for '%') for metric name

[Sriharsha Chintalapani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48065/#review136402
---


Ship it!




Ship It!

- Sriharsha Chintalapani


On June 3, 2016, 12:43 a.m., Jungtaek Lim wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48065/
> ---
> 
> (Updated June 3, 2016, 12:43 a.m.)
> 
> 
> Review request for Ambari, Sriharsha Chintalapani and Sid Wagle.
> 
> 
> Bugs: AMBARI-16949
> https://issues.apache.org/jira/browse/AMBARI-16949
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When we request '/ws/v1/timeline/metrics' with metric name which contains %25 
> (escaped '%'), response for the API is json describing there's NPE.
> And NPE is logged for ambari-metrics-collector log file.
> 
> ```
> 2016-05-30 09:15:05,061 WARN 
> org.apache.hadoop.yarn.webapp.GenericExceptionHandler: INTERNAL_SERVER_ERROR
> java.lang.NullPointerException
>   at 
> org.apache.hadoop.yarn.server.applicationhistoryservice.metrics.timeline.PhoenixHBaseAccessor.appendAggregateMetricFromResultSet(PhoenixHBaseAccessor.java:810)
>   at 
> org.apache.hadoop.yarn.server.applicationhistoryservice.metrics.timeline.PhoenixHBaseAccessor.getAggregateMetricRecords(PhoenixHBaseAccessor.java:772)
>   at 
> org.apache.hadoop.yarn.server.applicationhistoryservice.metrics.timeline.HBaseTimelineMetricStore.getTimelineMetrics(HBaseTimelineMetricStore.java:178)
>   at 
> org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.TimelineWebServices.getTimelineMetrics(TimelineWebServices.java:372)
>   at sun.reflect.GeneratedMethodAccessor21.invoke(Unknown Source)
> ```
> 
> Reason of NPE: 
> 
> Metrics are properly fetched with wildcard. But when applying functions to 
> result set, actual metric name is not exist from map of metric name to list 
> of function.
> 
> 
> Diffs
> -
> 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java
>  47962cb 
> 
> Diff: https://reviews.apache.org/r/48065/diff/
> 
> 
> Testing
> ---
> 
> Test failed from local but it's occurred from another modules, which is not 
> related to current modification.
> 
> ```
> Results :
> 
> Failed tests:
>   PrivilegeEventCreatorTest.putTest:107 expected:<...), Roles(
> Permission[2:
>   Users: testuser2
> Permission1:
>   Users: testuser
>   Groups: testgroup])> but was:<...), Roles(
> Permission[1:
>   Users: testuser
>   Groups: testgroup
> Permission2:
>   Users: testuser2])>
>   RepositoryVersionEventCreatorTest.postTest:70 expected:<...ating system: 
> redhat[6
> Repository ID(2), Repository name(MyRepo6), Base url(http://example6.com)
> Operating system: redhat7
> Repository ID(1), Repository name(MyRepo), Base url(http://example].com)
> )> but was:<...ating system: redhat[7
> Repository ID(1), Repository name(MyRepo), Base url(http://example.com)
> Operating system: redhat6
> Repository ID(2), Repository name(MyRepo6), Base url(http://example6].com)
> )>
>   RepositoryVersionEventCreatorTest.putTest:100 expected:<...ating system: 
> redhat[6
> Repository ID(2), Repository name(MyRepo6), Base url(http://example6.com)
> Operating system: redhat7
> Repository ID(1), Repository name(MyRepo), Base url(http://example].com)
> )> but was:<...ating system: redhat[7
> Repository ID(1), Repository name(MyRepo), Base url(http://example.com)
> Operating system: redhat6
> Repository ID(2), Repository name(MyRepo6), Base url(http://example6].com)
> )>
>   ViewPrivilegeEventCreatorTest.putTest:85 expected:<...tatus(200 OK), 
> Type([MyView), Version(MyView), Name(MyView), Permissions(
> Permission2:
>   Users: testuser2
> Permission1:
>   Users: testuser
>   Groups: testgroup])> but was:<...tatus(200 OK), Type([null), Version(null), 
> Name(null), Permissions(
> Permission1:
>   Users: testuser
>   Groups: testgroup
> Permission2:
>   Users: testuser2])>
>   
> ComponentResourceProviderTest.testGetResourcesAsAdministrator:190->testGetResources:296
>  expected:<[tru]e> but was:<[fals]e>
>   
> ComponentResourceProviderTest.testGetResourcesAsClusterAdministrator:195->testGetResources:296
>  expected:<[tru]e> but was:<[fals]e>
>   
> ComponentResourceProviderTest.testGetResourcesAsServiceAdministrator:200->testGetResources:296
>  expected:<[tru]e> but was:<[fals]e>
>   
> AmbariLdapDataPopulatorTest.testSynchronizeExistingLdapGroups_removeDuringIteration:333
>   Expectation failure on verify:
> AmbariLdapDataPopulatorTestInstance.getLdapGroupByMemberAttr("group2"): 
> expected: 1, actual: 0
>   UpgradeCatalog222Test.testInitializeStromAndKafkaWidgets:1107
>   Unexpected method call 
> 

Re: Review Request 48030: AMBARI-16946 Storm Metrics Sink has high chance to discard some datapoints

[Sriharsha Chintalapani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48030/#review136400
---


Ship it!




Ship It!

- Sriharsha Chintalapani


On May 30, 2016, 8:21 a.m., Jungtaek Lim wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48030/
> ---
> 
> (Updated May 30, 2016, 8:21 a.m.)
> 
> 
> Review request for Ambari, Sriharsha Chintalapani and Sid Wagle.
> 
> 
> Bugs: AMBARI-16946
> https://issues.apache.org/jira/browse/AMBARI-16946
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> There's a mismatch between TimelineMetricsCache and Storm metrics unit, while 
> TimelineMetricsCache considers "metric name + timestamp" to be unique but 
> Storm is not.
> 
> For example, assume that bolt B has task T1, T2 and B has registered metrics 
> M1. It's possible for metrics sink to receive (T1, M1) and (T2, M1) with same 
> timestamp TS1 (in TaskInfo, not current time), and received later will be 
> discarded from TimelineMetricsCache.
> 
> If we want to have unique metric point of Storm, we should use "topology name 
> + component name + task id + metric name" to metric name so that "metric name 
> + timestamp" will be unique.
> 
> There're other issues I would like to address, too.
> 
> - Currently, hostname is written to hostname of the machine which runs 
> metrics sink. Since TaskInfo has hostname of the machine which runs task, 
> we're better to use this.
> - Unit of timestamp of TaskInfo is second, while Storm Metrics Sink uses this 
> as millisecond, resulting in timestamp flaw, and malfunction of cache 
> eviction. It should be multiplied by 1000.
> - 'component name' is not unique across the cluster, so it's not fit for app 
> id. 'topology name' is unique so proper value of app id is topology name.
> 
> Consideration: Hostname for determining 'write shard' is set to hostname of 
> the machine which runs metrics sink. Since I don't know read also be sharded, 
> I'm not sure it's safe to use TaskInfo's hostname to hostname of 
> TimelineMetric. Please review carefully regarding this.
> 
> 
> Diffs
> -
> 
>   
> ambari-metrics/ambari-metrics-storm-sink/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSink.java
>  02f5598 
>   
> ambari-metrics/ambari-metrics-storm-sink/src/test/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSinkTest.java
>  8171a4d 
> 
> Diff: https://reviews.apache.org/r/48030/diff/
> 
> 
> Testing
> ---
> 
> I tested this only ambari-metrics module since changeset is not related on 
> other modules.
> 
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO]
> [INFO] ambari-metrics . SUCCESS [  0.896 
> s]
> [INFO] Ambari Metrics Common .. SUCCESS [ 13.386 
> s]
> [INFO] Ambari Metrics Hadoop Sink . SUCCESS [  5.085 
> s]
> [INFO] Ambari Metrics Flume Sink .. SUCCESS [  6.827 
> s]
> [INFO] Ambari Metrics Kafka Sink .. SUCCESS [  4.190 
> s]
> [INFO] Ambari Metrics Storm Sink .. SUCCESS [  1.384 
> s]
> [INFO] Ambari Metrics Collector ... SUCCESS [04:06 
> min]
> [INFO] Ambari Metrics Monitor . SUCCESS [  3.556 
> s]
> [INFO] Ambari Metrics Grafana . SUCCESS [01:03 
> min]
> [INFO] Ambari Metrics Assembly  SUCCESS [  3.567 
> s]
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 05:48 min
> [INFO] Finished at: 2016-05-30T16:46:07+09:00
> [INFO] Final Memory: 78M/1038M
> [INFO] 
> 
> 
> In fact, I tried to run `mvn test` from ambari root directory but build is 
> failing from ambari-web.
> 
> ```
> > fsevents@0.2.1 install 
> > /Users/jlim/WorkArea/JavaProjects/ambari/ambari-web/node_modules/chokidar/node_modules/fsevents
> > node-gyp rebuild
> ...
> npm WARN install:fsevents fsevents@0.2.1 install: `node-gyp rebuild`
> npm WARN install:fsevents Exit status 1
> ```
> 
> No luck on `npm install`, too.
> 
> 
> Thanks,
> 
> Jungtaek Lim
> 
>

Review Request 48309: AMBARI-17047: Firewall check returns WARNING even if iptables and firewalld are stopped on CentOS7

[Masahiro Tanaka]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48309/
---

Review request for Ambari, Andrew Onischuk, Dmytro Sen, and Florian Barca.


Bugs: AMBARI-17047
https://issues.apache.org/jira/browse/AMBARI-17047


Repository: ambari


Description
---

In firewall.py, `systemctl is-active iptables || systemctl is-active firewalld` 
is passed to `run_in_shell` function, which splits cmd string by using 
`shlex.split`.

run_in_shell function finally calls `subprocess.Popen` with `shell=True`, so 
the cmd string is evaluated like `Popen(['/bin/sh', '-c', 'systemctl', 
'is-active', 'iptables', '||', 'systemctl', 'is-active', 'firewalld'])`. This 
doesn't returns values as expected, because after args[1] (in this case, after 
the first `is-active`) are evaluated as sh arguements.

`systemctl is-active` can take multiple arugments, so we can use it.


Diffs
-

  ambari-common/src/main/python/ambari_commons/firewall.py 72e6d26 

Diff: https://reviews.apache.org/r/48309/diff/


Testing
---

mvn clean test & manual test


Thanks,

Masahiro Tanaka

Re: Review Request 48276: View config-Custom views are not working after kerberization

[Pallav Kulshreshtha]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48276/#review136398
---


Ship it!




Ship It!

- Pallav Kulshreshtha


On June 6, 2016, 12:47 p.m., Gaurav Nagar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48276/
> ---
> 
> (Updated June 6, 2016, 12:47 p.m.)
> 
> 
> Review request for Ambari, DIPAYAN BHOWMICK, Nitiraj Rathore, Pallav 
> Kulshreshtha, Rohit Choudhary, and Ashwin Rajeev.
> 
> 
> Bugs: AMBARI-17056
> https://issues.apache.org/jira/browse/AMBARI-17056
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Fixed setting of webhdfs auth for custom views.
> 
> 
> Diffs
> -
> 
>   
> contrib/views/hive/src/main/java/org/apache/ambari/view/hive/client/ConnectionFactory.java
>  0eb7906 
>   
> contrib/views/utils/src/main/java/org/apache/ambari/view/utils/hdfs/AuthConfigurationBuilder.java
>  10ec1fb 
>   
> contrib/views/utils/src/main/java/org/apache/ambari/view/utils/hdfs/ConfigurationBuilder.java
>  9edf9d7 
> 
> Diff: https://reviews.apache.org/r/48276/diff/
> 
> 
> Testing
> ---
> 
> Manually tested
> 
> 
> Thanks,
> 
> Gaurav Nagar
> 
>

Re: Review Request 48274: Show better error message when VIEW USER can not access CAPACITY SCHEDULER View

[Pallav Kulshreshtha]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48274/#review136397
---


Ship it!




Ship It!

- Pallav Kulshreshtha


On June 6, 2016, 12:23 p.m., Gaurav Nagar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48274/
> ---
> 
> (Updated June 6, 2016, 12:23 p.m.)
> 
> 
> Review request for Ambari, DIPAYAN BHOWMICK, Nitiraj Rathore, Pallav 
> Kulshreshtha, Rohit Choudhary, and Ashwin Rajeev.
> 
> 
> Bugs: AMBARI-17055
> https://issues.apache.org/jira/browse/AMBARI-17055
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Added error message when user does not have read permission on configurations.
> Added message when view is in read only mode.
> 
> 
> Diffs
> -
> 
>   
> contrib/views/capacity-scheduler/src/main/java/org/apache/ambari/view/capacityscheduler/ConfigurationService.java
>  42076fe 
>   
> contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/queues.hbs
>  94cb421 
>   
> contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/queues/index.hbs
>  1c468b5 
>   
> contrib/views/utils/src/main/java/org/apache/ambari/view/utils/ambari/AmbariApi.java
>  86d8434 
> 
> Diff: https://reviews.apache.org/r/48274/diff/
> 
> 
> Testing
> ---
> 
> Manully Tested
> 
> 
> Thanks,
> 
> Gaurav Nagar
> 
>

Re: Review Request 47814: Capacity Scheduler View - Save only, Save and Restart Rm and Delete queue integrations

[Pallav Kulshreshtha]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47814/#review136396
---


Ship it!




Ship It!

- Pallav Kulshreshtha


On May 25, 2016, 6:07 a.m., Akhil PB wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47814/
> ---
> 
> (Updated May 25, 2016, 6:07 a.m.)
> 
> 
> Review request for Ambari, DIPAYAN BHOWMICK, Pallav Kulshreshtha, and 
> Sreenath Somarajapuram.
> 
> 
> Bugs: AMBARI-16842
> https://issues.apache.org/jira/browse/AMBARI-16842
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Save and Restart Rm, and Delete queue integrations in new capacity scheduler 
> view.
> 
> 
> Diffs
> -
> 
>   
> contrib/views/capacity-scheduler/src/main/resources/ui/app/components/queueBadge.js
>  1e2b77c 
>   
> contrib/views/capacity-scheduler/src/main/resources/ui/app/components/queueSummary.js
>  3d4f7be 
>   
> contrib/views/capacity-scheduler/src/main/resources/ui/app/controllers/editqueue.js
>  e981ea5 
>   
> contrib/views/capacity-scheduler/src/main/resources/ui/app/controllers/queuesconf.js
>  14ea85c 
>   contrib/views/capacity-scheduler/src/main/resources/ui/app/models/queue.js 
> 46f14ad 
>   
> contrib/views/capacity-scheduler/src/main/resources/ui/app/styles/application.less
>  89b8eb7 
>   
> contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/capsched/queuesconf.hbs
>  3837a03 
>   
> contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/capsched/scheduler.hbs
>  364bff2 
>   
> contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/components/queueSummary.hbs
>  7ba9e62 
>   
> contrib/views/capacity-scheduler/src/main/resources/ui/app/views/editQueueCapacity.js
>  5c3793f 
>   
> contrib/views/capacity-scheduler/src/main/resources/ui/app/views/queuesconf.js
>  7cb71a7 
> 
> Diff: https://reviews.apache.org/r/47814/diff/
> 
> 
> Testing
> ---
> 
> Manual Unit Testing
> 
> 
> Thanks,
> 
> Akhil PB
> 
>

Review Request 48308: [AMBARI-17078] Make Spark2-ThriftServer and Livy Server as optional by default

[Jeff Zhang]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48308/
---

Review request for Ambari, Jayush Luniya and Sumit Mohanty.


Bugs: AMBARI-17078
https://issues.apache.org/jira/browse/AMBARI-17078


Repository: ambari


Description
---

Spark2-ThriftServer and Livy Server are not being selected by default, but 
should be unselected by default.


Diffs
-

  ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 
36fe066b65b12326a128fdb5e8caf9ad56a8211c 

Diff: https://reviews.apache.org/r/48308/diff/


Testing
---

Manullay verified


Thanks,

Jeff Zhang

Re: Review Request 48229: Refactor service_advisor apis to remove passing of stack_advisor

[Lav Jain]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48229/
---

(Updated June 7, 2016, 2:53 a.m.)


Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, Jayush 
Luniya, Matt, and Tim Thorpe.


Changes
---

Rebased to branch-2.4


Bugs: AMBARI-16663
https://issues.apache.org/jira/browse/AMBARI-16663


Repository: ambari


Description
---

The original intent was to move the common functions that are used by both 
stack and service advisors to a separate utils file to avoid passing 
stack_advisor class into service advisor apis.


Diffs (updated)
-

  
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py 
2c78bf1 
  ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py 
9b34171 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
1dedf3d 
  ambari-server/src/main/resources/stacks/service_advisor.py 3d6c293 
  ambari-server/src/main/resources/stacks/stack_advisor.py beba225 
  ambari-server/src/test/python/stacks/2.3/HAWQ/test_service_advisor.py 384b8be 
  ambari-server/src/test/python/stacks/2.3/PXF/test_service_advisor.py c3a63cc 
  ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py caf121c 

Diff: https://reviews.apache.org/r/48229/diff/


Testing
---

Lavs-MacBook-Pro:common ljain$ python -m discover -v
test_createComponentLayoutRecommendations_hawq_1_Host 
(test_stack_advisor.TestHDP23StackAdvisor) ... ServiceAdvisor implementation 
for service HAWQ was loaded
[u'c6401.ambari.apache.org']
[u'c6401.ambari.apache.org']
ok
test_createComponentLayoutRecommendations_hawq_3_Hosts 
(test_stack_advisor.TestHDP23StackAdvisor)
Test that HAWQSTANDBY is recommended on a 3-node cluster ... ServiceAdvisor 
implementation for service HAWQ was loaded
[u'c6401.ambari.apache.org', u'c6402.ambari.apache.org', 
u'c6403.ambari.apache.org']
[u'c6401.ambari.apache.org', u'c6402.ambari.apache.org', 
u'c6403.ambari.apache.org']
ok
test_createComponentLayoutRecommendations_hawqsegment_add_service_wizard_already_installed
 (test_stack_advisor.TestHDP23StackAdvisor)
Test that HAWQSEGMENT does not get recommended during Add Service Wizard, when 
HAWQ has already been installed ... ServiceAdvisor implementation for service 
HAWQ was loaded
ok
test_createComponentLayoutRecommendations_hawqsegment_add_service_wizard_to_be_installed
 (test_stack_advisor.TestHDP23StackAdvisor)
Test that HAWQSEGMENT gets recommended correctly during Add Service Wizard, 
when HAWQ is selected for installation ... ServiceAdvisor implementation for 
service HAWQ was loaded
ok
test_createComponentLayoutRecommendations_hawqsegment_cluster_install 
(test_stack_advisor.TestHDP23StackAdvisor)
Test that HAWQSEGMENT gets recommended correctly during Cluster Install Wizard, 
when HAWQ is selected for installation ... ServiceAdvisor implementation for 
service HAWQ was loaded
ok
test_createComponentLayoutRecommendations_no_hawq_3_Hosts 
(test_stack_advisor.TestHDP23StackAdvisor)
Test no failures when there are no HAWQ components ... ok
test_createComponentLayoutRecommendations_pxf_add_service_wizard_already_installed
 (test_stack_advisor.TestHDP23StackAdvisor)
Test that PXF does not get recommended during Add Service Wizard, when PXF has 
already been installed ... ServiceAdvisor implementation for service PXF was 
loaded
ok
test_createComponentLayoutRecommendations_pxf_add_service_wizard_to_be_installed
 (test_stack_advisor.TestHDP23StackAdvisor)
Test that PXF gets recommended correctly during Add Service Wizard, when PXF is 
selected for installation ... ServiceAdvisor implementation for service PXF was 
loaded
ok
test_createComponentLayoutRecommendations_pxf_cluster_install 
(test_stack_advisor.TestHDP23StackAdvisor)
Test that PXF gets recommended correctly during Cluster Install Wizard, when 
PXF is selected for installation ... ServiceAdvisor implementation for service 
PXF was loaded
ok
test_getComponentLayoutValidations_hawq_3_Hosts 
(test_stack_advisor.TestHDP23StackAdvisor)
Test layout validations for HAWQ components on a 3-node cluster ... 
ServiceAdvisor implementation for service HAWQ was loaded
ok
test_getComponentLayoutValidations_hawqsegment_not_co_located_with_datanode 
(test_stack_advisor.TestHDP23StackAdvisor)
Test validation warning for HAWQ segment not colocated with DATANODE ... 
ServiceAdvisor implementation for service HAWQ was loaded
ok
test_getComponentLayoutValidations_nohawq_3_Hosts 
(test_stack_advisor.TestHDP23StackAdvisor)
Test no failures when there are no HAWQ components on a 3-node cluster ... ok
test_getComponentLayoutValidations_pxf_co_located_with_nn_and_dn 
(test_stack_advisor.TestHDP23StackAdvisor)
Test NO warning is generated when PXF is co-located with NAMENODE and DATANODE 
... ServiceAdvisor implementation for service PXF was loaded
ok

Re: Review Request 48290: Change HAWQ DFS Url from hawq_default to hawq_data

[Lav Jain]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48290/#review136395
---


Ship it!




Ship It!

- Lav Jain


On June 6, 2016, 7:17 p.m., Matt wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48290/
> ---
> 
> (Updated June 6, 2016, 7:17 p.m.)
> 
> 
> Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, and Lav 
> Jain.
> 
> 
> Bugs: AMBARI-17070
> https://issues.apache.org/jira/browse/AMBARI-17070
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Change HAWQ DFS Url from hawq_default to hawq_data
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-site.xml
>  3ac4e89 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/params.py
>  a8a4487 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
>  ec04fd2 
>   ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqmaster.py 70d7ed1 
>   ambari-server/src/test/python/stacks/2.3/configs/hawq_default.json b839de6 
>   ambari-web/app/data/HDP2/ha_properties.js 18c5a68 
>   
> ambari-web/test/controllers/main/admin/highAvailability/nameNode/step3_controller_test.js
>  ca55a32 
>   ambari-web/test/utils/configs/config_initializer_test.js 48eae23 
> 
> Diff: https://reviews.apache.org/r/48290/diff/
> 
> 
> Testing
> ---
> 
> Waiting for results from Hadoop QA
> 
> 
> Thanks,
> 
> Matt
> 
>

Review Request 48306: AMBARI-17077 - Unable to change user role in list view

[Richard Zang]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48306/
---

Review request for Ambari and Zhe (Joe) Wang.


Bugs: AMBARI-17077
https://issues.apache.org/jira/browse/AMBARI-17077


Repository: ambari


Description
---

Fix role update logic


Diffs
-

  
ambari-admin/src/main/resources/ui/admin-web/app/scripts/controllers/clusters/UserAccessListCtrl.js
 cd72ca6 

Diff: https://reviews.apache.org/r/48306/diff/


Testing
---

Manually tested on live cluster.
All unit tests passed.
PhantomJS 1.9.7 (Mac OS X): Executed 71 of 71 SUCCESS (0.151 secs / 0.399 secs)


Thanks,

Richard Zang

Re: Review Request 48301: AMBARI-17075. Set 'hive.server2.tez.default.queues' config value to be the one used by config hive.llap.daemon.queue.name for 'llap' app.

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48301/#review136394
---


Ship it!




Ship It!

- Sumit Mohanty


On June 7, 2016, 12:24 a.m., Swapan Shridhar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48301/
> ---
> 
> (Updated June 7, 2016, 12:24 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17075
> https://issues.apache.org/jira/browse/AMBARI-17075
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> - Currently, 'hive.server2.tez.default.queues' config value is set to 
> 'default'.
> - Need to update it with the value of 'hive.llap.daemon.queue.name'.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-interactive-site.xml
>  4133e5b 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
> 379c268 
>   ambari-server/src/test/python/stacks/2.5/common/test_stack_advisor.py 
> 46e9f47 
> 
> Diff: https://reviews.apache.org/r/48301/diff/
> 
> 
> Testing
> ---
> 
> - Added new UT's and updated the relevant existing one.
> 
> Relevant Python UT passes.
> 
> - Errors seen in other UTs (87 total).
> 
> 
> Thanks,
> 
> Swapan Shridhar
> 
>

Re: Review Request 48296: AMBARI-17073 HDF Management install does not allow one to select public repo

[Zhe (Joe) Wang]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48296/
---

(Updated June 7, 2016, 12:36 a.m.)


Review request for Ambari, Jaimin Jetly, Nate Cole, Richard Zang, Xi Wang, and 
Yusaku Sako.


Bugs: AMBARI-17073
https://issues.apache.org/jira/browse/AMBARI-17073


Repository: ambari


Description
---

To fulfill the requirement of enforcing users to choose local repo when they 
don’t have internet access, in FE we use the logic of checking whether all 
versions are stack_default versions. That logic is true for HDP for now, but 
not for HDF.


Diffs (updated)
-

  ambari-web/app/messages.js e8ca9ae 
  ambari-web/app/templates/wizard/step1.hbs 72a2099 
  ambari-web/app/templates/wizard/step1/public_option_disabled_window_body.hbs 
25cf49f 
  ambari-web/app/views/wizard/step1_view.js 503d5ef 

Diff: https://reviews.apache.org/r/48296/diff/


Testing
---

Local ambari-web test passed.
28646 tests complete (24 seconds)
154 tests pending
Manual testing done.


Thanks,

Zhe (Joe) Wang

Review Request 48301: AMBARI-17075. Set 'hive.server2.tez.default.queues' config value to be the one used by config hive.llap.daemon.queue.name for 'llap' app.

[Swapan Shridhar]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48301/
---

Review request for Ambari, Alejandro Fernandez and Sumit Mohanty.


Bugs: AMBARI-17075
https://issues.apache.org/jira/browse/AMBARI-17075


Repository: ambari


Description
---

- Currently, 'hive.server2.tez.default.queues' config value is set to 'default'.
- Need to update it with the value of 'hive.llap.daemon.queue.name'.


Diffs
-

  
ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-interactive-site.xml
 4133e5b 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
379c268 
  ambari-server/src/test/python/stacks/2.5/common/test_stack_advisor.py 46e9f47 

Diff: https://reviews.apache.org/r/48301/diff/


Testing
---

- Added new UT's and updated the relevant existing one.

Relevant Python UT passes.

- Errors seen in other UTs (87 total).


Thanks,

Swapan Shridhar

Re: Review Request 48288: Microsoft-r install script should use Ambari libraries instead of direct shell calls

[Srimanth Gunturi]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48288/#review136385
---


Ship it!




Ship It!

- Srimanth Gunturi


On June 6, 2016, 5:30 p.m., Balázs Bence Sári wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48288/
> ---
> 
> (Updated June 6, 2016, 5:30 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jayush Luniya, Srimanth 
> Gunturi, Sumit Gupta, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-17067
> https://issues.apache.org/jira/browse/AMBARI-17067
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Made the following changes:
> - Use Ambari Python API only
> - Use new rpm's
> - Cleaned up imports.
> - Added supported HDP versions to the mpack
> 
> 
> Diffs
> -
> 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/package/scripts/microsoft_r.py
>  5b6e482 
>   contrib/management-packs/microsoft-r_mpack/src/main/resources/mpack.json 
> 17c2f54 
> 
> Diff: https://reviews.apache.org/r/48288/diff/
> 
> 
> Testing
> ---
> 
> On a a single node cluster (CentOS 6.4, Ambari 2.2 upgraded to 2.4 snapshot 
> build, HDP 2.3), did the following testing:
> - Built the management pack with mvn install
> - Installed it on the Ambari Server by running this command: ambari-server 
> install-mpack --mpack microsoft-r_mpack-1.0.0.0-SNAPSHOT.tar.gz
> - Restarted the Ambari Server
> - On the UI, chose add service and selected Microsoft R.
> - Install completed without errors. It had a warning due to missing service 
> check script (covered by another Jira ticket).
> 
> 
> Thanks,
> 
> Balázs Bence Sári
> 
>

Re: Review Request 48293: Make QuickLink to HS2 UI work when HTTPS is enabled

[Yusaku Sako]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48293/#review136378
---


Ship it!




Ship It!

- Yusaku Sako


On June 6, 2016, 9:24 p.m., Jaimin Jetly wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48293/
> ---
> 
> (Updated June 6, 2016, 9:24 p.m.)
> 
> 
> Review request for Ambari, Zhe (Joe) Wang, Richard Zang, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-17072
> https://issues.apache.org/jira/browse/AMBARI-17072
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> There are a two important hive-site.xml parameters that govern the url:
> i) hive.server2.webui.port : This is the port on which the webui hosts itself.
> ii) hive.server2.webui.use.ssl : Whether or not ssl is used.
> If ssl is used, we use https:// and if not, we use http:// . Either way, the 
> service seems to be intended to be hosted at :/llap.html
> this will work only for hive2 and not hive1.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-interactive-site.xml
>  8e32e9dd 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/quicklinks/quicklinks.json
>  e586d20 
> 
> Diff: https://reviews.apache.org/r/48293/diff/
> 
> 
> Testing
> ---
> 
> Tested the patch manualy on the cluster.
> unit tests not required as only metadata (quicklinks.json) is cahnged as part 
> of the patch.
> 
> 
> Thanks,
> 
> Jaimin Jetly
> 
>

Re: Review Request 48293: Make QuickLink to HS2 UI work when HTTPS is enabled

[Yusaku Sako]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48293/#review136377
---




ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-interactive-site.xml
 (line 263)


I had some concerns about this "on-ambari-upgrade", "on-stack-upgrade", 
etc., but this is a separate issue that others have addressed (e.g., 
https://reviews.apache.org/r/48212/#comment201053)


- Yusaku Sako


On June 6, 2016, 9:24 p.m., Jaimin Jetly wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48293/
> ---
> 
> (Updated June 6, 2016, 9:24 p.m.)
> 
> 
> Review request for Ambari, Zhe (Joe) Wang, Richard Zang, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-17072
> https://issues.apache.org/jira/browse/AMBARI-17072
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> There are a two important hive-site.xml parameters that govern the url:
> i) hive.server2.webui.port : This is the port on which the webui hosts itself.
> ii) hive.server2.webui.use.ssl : Whether or not ssl is used.
> If ssl is used, we use https:// and if not, we use http:// . Either way, the 
> service seems to be intended to be hosted at :/llap.html
> this will work only for hive2 and not hive1.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-interactive-site.xml
>  8e32e9dd 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/quicklinks/quicklinks.json
>  e586d20 
> 
> Diff: https://reviews.apache.org/r/48293/diff/
> 
> 
> Testing
> ---
> 
> Tested the patch manualy on the cluster.
> unit tests not required as only metadata (quicklinks.json) is cahnged as part 
> of the patch.
> 
> 
> Thanks,
> 
> Jaimin Jetly
> 
>

Re: Review Request 48292: VDF: exception when trying to register -> add versions

[Nate Cole]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48292/
---

(Updated June 6, 2016, 5:34 p.m.)


Review request for Ambari, Alejandro Fernandez, Dmitro Lisnichenko, and 
Jonathan Hurley.


Bugs: AMBARI-17071
https://issues.apache.org/jira/browse/AMBARI-17071


Repository: ambari


Description
---

Only verify duplicate repo URLs if Ambari is managing the repos for the desired 
repo_version.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RepositoryVersionResourceProvider.java
 57fb115 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/RepositoryVersionResourceProviderTest.java
 a61af95 

Diff: https://reviews.apache.org/r/48292/diff/


Testing (updated)
---

Manual.  Automated:


Tests run: 4454, Failures: 0, Errors: 0, Skipped: 34

[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 36:27.253s
[INFO] Finished at: Mon Jun 06 17:26:34 EDT 2016
[INFO] Final Memory: 35M/569M
[INFO] 


Thanks,

Nate Cole

Re: Review Request 48212: Fix files mentioned by ServicePropertiesTest on latest commits

[Jayush Luniya]

> On June 3, 2016, 5:16 p.m., Jayush Luniya wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-conf.xml,
> >  line 32
> > 
> >
> > Why are these mandatory property attributes to cause UT failures? 
> > Shouldnt they have defaults if not defined?
> > 
> > Also how do we decide what to set these properties too? (i.e. 
> > on-ambari-upgrade add="false" v/s on-ambari-upgrade add="true") Can you 
> > provide examples? 
> > 
> > I am not clear on why on-ambari-upgrade delete="true" and 
> > on-stack-upgrade delete="false"?
> > 
> > I will look at your original patch to get some context and then review 
> > later today.
> 
> Dmitro Lisnichenko wrote:
> the idea is to force developers to specify what to do during upgrade 
> explicitly when adding properties, instead of trying to guess correct policy 
> during upgrade
> Regarding documentation - as of now patch causes troubles during RU/EU, 
> and may be partially reverted soon. So let's return to this question later

+1'ed so that we can fix this issue, but we still should get this documented.


- Jayush


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48212/#review136078
---


On June 3, 2016, 4:55 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48212/
> ---
> 
> (Updated June 3, 2016, 4:55 p.m.)
> 
> 
> Review request for Ambari and Jayush Luniya.
> 
> 
> Bugs: AMBARI-17032
> https://issues.apache.org/jira/browse/AMBARI-17032
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> {code}
> ServicePropertiesTest.validatePropertySchemaOfServiceXMLs:50 » Ambari File 
> /tm...
> {code}
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-conf.xml
>  d0acdda 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-env.xml
>  410a1c1 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-log4j-properties.xml
>  d84207a 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-spark-blacklist.xml
>  4a5fbfb 
> 
> Diff: https://reviews.apache.org/r/48212/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Re: Review Request 48212: Fix files mentioned by ServicePropertiesTest on latest commits

[Jayush Luniya]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48212/#review136366
---


Ship it!




Ship It!

- Jayush Luniya


On June 3, 2016, 4:55 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48212/
> ---
> 
> (Updated June 3, 2016, 4:55 p.m.)
> 
> 
> Review request for Ambari and Jayush Luniya.
> 
> 
> Bugs: AMBARI-17032
> https://issues.apache.org/jira/browse/AMBARI-17032
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> {code}
> ServicePropertiesTest.validatePropertySchemaOfServiceXMLs:50 » Ambari File 
> /tm...
> {code}
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-conf.xml
>  d0acdda 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-env.xml
>  410a1c1 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-log4j-properties.xml
>  d84207a 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-spark-blacklist.xml
>  4a5fbfb 
> 
> Diff: https://reviews.apache.org/r/48212/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Review Request 48293: Make QuickLink to HS2 UI work when HTTPS is enabled

[Jaimin Jetly]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48293/
---

Review request for Ambari, Zhe (Joe) Wang, Richard Zang, and Yusaku Sako.


Bugs: AMBARI-17072
https://issues.apache.org/jira/browse/AMBARI-17072


Repository: ambari


Description
---

There are a two important hive-site.xml parameters that govern the url:
i) hive.server2.webui.port : This is the port on which the webui hosts itself.
ii) hive.server2.webui.use.ssl : Whether or not ssl is used.
If ssl is used, we use https:// and if not, we use http:// . Either way, the 
service seems to be intended to be hosted at :/llap.html
this will work only for hive2 and not hive1.


Diffs
-

  
ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-interactive-site.xml
 8e32e9dd 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/quicklinks/quicklinks.json
 e586d20 

Diff: https://reviews.apache.org/r/48293/diff/


Testing
---

Tested the patch manualy on the cluster.
unit tests not required as only metadata (quicklinks.json) is cahnged as part 
of the patch.


Thanks,

Jaimin Jetly

Re: Review Request 48235: Show only relevant properties in HAWQ based on the status of HAWQ Resource Manager type

[bhuvnesh chaudhary]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48235/
---

(Updated June 6, 2016, 9:23 p.m.)


Review request for Ambari, Alexander Denissov, Lav Jain, and Matt.


Changes
---

Applied changes


Bugs: AMBARI-17023
https://issues.apache.org/jira/browse/AMBARI-17023


Repository: ambari


Description
---

Show only relevant properties based on the status of HAWQ Resource Manager.
When Yarn mode is enable, only two parameters should be invisible.
hawq_rm_memory_limit_perseg
hawq_rm_nvcore_limit_perseg
When standalone mode is enabled, four parameters should be invisible. what you 
list are correct.
hawq_rm_yarn_app_name
hawq_rm_yarn_queue_name
hawq_rm_yarn_scheduler_address
hawq_rm_yarn_address


Diffs (updated)
-

  
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-site.xml
 3ac4e89 
  
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py 
28eb82f 
  ambari-server/src/test/python/stacks/2.3/HAWQ/test_service_advisor.py 5750938 
  ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 424a386 

Diff: https://reviews.apache.org/r/48235/diff/


Testing
---

yes.
Now testing 
/Users/bhuvneshchaudhary/github/ambari-toolbox/vagrant/ambari/ambari-server/src/test/python/stacks/2.3/HAWQ
~/github/ambari-toolbox/vagrant/ambari/ambari-server/src/test/python/stacks/2.3/HAWQ
 ~/github/ambari-toolbox/vagrant/ambari
test_hawq_master_critical 
(test_alert_component_status.TestAlertComponentStatus) ... ok
test_hawq_master_ok (test_alert_component_status.TestAlertComponentStatus) ... 
ok
test_hawq_segment_critical 
(test_alert_component_status.TestAlertComponentStatus) ... ok
test_hawq_segment_ok (test_alert_component_status.TestAlertComponentStatus) ... 
ok
test_hawq_standby_critical 
(test_alert_component_status.TestAlertComponentStatus) ... ok
test_hawq_standby_ok (test_alert_component_status.TestAlertComponentStatus) ... 
ok
test_missing_configs (test_alert_component_status.TestAlertComponentStatus) ... 
ok
test_exception_registration_status 
(test_alert_segment_registration_status.TestAlertRegistrationStatus) ... No 
handlers could be found for logger "ambari_alerts"
ok
test_missing_configs 
(test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
test_missing_slave_file 
(test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
test_successful_registration_status 
(test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
test_unsuccessful_empty_db_registration_status 
(test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
test_unsuccessful_registration_status 
(test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
test_unsuccessful_registration_status_plural 
(test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
test_missing_configs (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_no_standby_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_none_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_not_configured_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_not_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_synchronizing_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_unknown_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_configure_default (test_hawqmaster.TestHawqMaster) ... ok
test_install_default (test_hawqmaster.TestHawqMaster) ... ok
test_remove_hawq_standby (test_hawqmaster.TestHawqMaster)
Run custom command Remove HAWQ Standby ... 2016-06-03 15:50:22,295 - Removing 
HAWQ Standby Master ...
ok
test_resync_hawq_standby (test_hawqmaster.TestHawqMaster)
Run custom command Resync HAWQ Standby ... 2016-06-03 15:50:22,301 - HAWQ 
Standby Master Re-Sync started in fast mode...
ok
test_run_hawq_check_case1 (test_hawqmaster.TestHawqMaster)
Running HAWQ Check Case 1: Non HDFS-HA, Standalone Resource Management, Not 
Kerberized ... 2016-06-03 15:50:22,307 - Executing HAWQ Check ...
ok
test_run_hawq_check_case10 (test_hawqmaster.TestHawqMaster)
Running HAWQ Check Case 10: HDFS-HA, YARN Resource Management Non YARN_HA, 
Kerberized ... 2016-06-03 15:50:22,321 - Executing HAWQ Check ...
ok
test_run_hawq_check_case11 (test_hawqmaster.TestHawqMaster)
Running HAWQ Check Case 11: HDFS-HA, YARN Resource Management YARN_HA, Not 
Kerberized ... 2016-06-03 15:50:22,333 - Executing HAWQ Check ...
ok
test_run_hawq_check_case12 (test_hawqmaster.TestHawqMaster)
Running HAWQ Check Case 12: HDFS-HA, YARN Resource Management YARN_HA, 
Kerberized ... 2016-06-03 15:50:22,346 - Executing HAWQ Check ...
ok
test_run_hawq_check_case2 (test_hawqmaster.TestHawqMaster)
Running HAWQ Check Case 2: Non HDFS-HA, 

Re: Review Request 48235: Show only relevant properties in HAWQ based on the status of HAWQ Resource Manager type

[bhuvnesh chaudhary]

> On June 3, 2016, 11:05 p.m., Alexander Denissov wrote:
> > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py,
> >  line 185
> > 
> >
> > better to move these lines higher to after line 149 where the similar 
> > logic resides ?

done


> On June 3, 2016, 11:05 p.m., Alexander Denissov wrote:
> > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py,
> >  line 193
> > 
> >
> > are we repeating default values already defined in .xml file ? Would be 
> > better not to do that, if possible.

fixed for vm.overcommit_ratio.


- bhuvnesh


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48235/#review136128
---


On June 3, 2016, 10:49 p.m., bhuvnesh chaudhary wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48235/
> ---
> 
> (Updated June 3, 2016, 10:49 p.m.)
> 
> 
> Review request for Ambari, Alexander Denissov, Lav Jain, and Matt.
> 
> 
> Bugs: AMBARI-17023
> https://issues.apache.org/jira/browse/AMBARI-17023
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Show only relevant properties based on the status of HAWQ Resource Manager.
> When Yarn mode is enable, only two parameters should be invisible.
> hawq_rm_memory_limit_perseg
> hawq_rm_nvcore_limit_perseg
> When standalone mode is enabled, four parameters should be invisible. what 
> you list are correct.
> hawq_rm_yarn_app_name
> hawq_rm_yarn_queue_name
> hawq_rm_yarn_scheduler_address
> hawq_rm_yarn_address
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-site.xml
>  3ac4e89 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py
>  28eb82f 
>   ambari-server/src/test/python/stacks/2.3/HAWQ/test_service_advisor.py 
> 5750938 
> 
> Diff: https://reviews.apache.org/r/48235/diff/
> 
> 
> Testing
> ---
> 
> yes.
> Now testing 
> /Users/bhuvneshchaudhary/github/ambari-toolbox/vagrant/ambari/ambari-server/src/test/python/stacks/2.3/HAWQ
> ~/github/ambari-toolbox/vagrant/ambari/ambari-server/src/test/python/stacks/2.3/HAWQ
>  ~/github/ambari-toolbox/vagrant/ambari
> test_hawq_master_critical 
> (test_alert_component_status.TestAlertComponentStatus) ... ok
> test_hawq_master_ok (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_hawq_segment_critical 
> (test_alert_component_status.TestAlertComponentStatus) ... ok
> test_hawq_segment_ok (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_hawq_standby_critical 
> (test_alert_component_status.TestAlertComponentStatus) ... ok
> test_hawq_standby_ok (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_missing_configs (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_exception_registration_status 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... No 
> handlers could be found for logger "ambari_alerts"
> ok
> test_missing_configs 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_missing_slave_file 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_successful_registration_status 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_unsuccessful_empty_db_registration_status 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_unsuccessful_registration_status 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_unsuccessful_registration_status_plural 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_missing_configs (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_no_standby_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_none_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_not_configured_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_not_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... 
> ok
> test_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_synchronizing_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_unknown_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_configure_default (test_hawqmaster.TestHawqMaster) ... ok
> test_install_default (test_hawqmaster.TestHawqMaster) ... ok
> test_remove_hawq_standby (test_hawqmaster.TestHawqMaster)
> Run custom command Remove HAWQ Standby ... 2016-06-03 15:50:22,295 - Removing 
> HAWQ 

Review Request 48292: VDF: exception when trying to register -> add versions

[Nate Cole]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48292/
---

Review request for Ambari, Alejandro Fernandez, Dmitro Lisnichenko, and 
Jonathan Hurley.


Bugs: AMBARI-17071
https://issues.apache.org/jira/browse/AMBARI-17071


Repository: ambari


Description
---

Only verify duplicate repo URLs if Ambari is managing the repos for the desired 
repo_version.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RepositoryVersionResourceProvider.java
 57fb115 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/RepositoryVersionResourceProviderTest.java
 a61af95 

Diff: https://reviews.apache.org/r/48292/diff/


Testing
---

Manual.  Automated pending.


Thanks,

Nate Cole

Re: Review Request 48258: Fix description of SERVICE.ADD_DELETE_SERVICES permission

[Nate Cole]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48258/#review136359
---


Ship it!




Ship It!

- Nate Cole


On June 6, 2016, 3:54 p.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48258/
> ---
> 
> (Updated June 6, 2016, 3:54 p.m.)
> 
> 
> Review request for Ambari, Denys Buzhor, Jonathan Robie, Nate Cole, and 
> Yusaku Sako.
> 
> 
> Bugs: AMBARI-17043
> https://issues.apache.org/jira/browse/AMBARI-17043
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> The description of the SERVICE.ADD_DELETE_SERVICES permission currently reads
> 
> ```
> Add Service to cluster
> ```
> 
> This should be changed to
> 
> ```
> Add/delete services
> ```
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog230.java
>  be9c2e2 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
>  01322b2 
>   ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql 940542d 
>   ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql eb2b349 
>   ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql de8c2e6 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 0a8d6c9 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql 
> 4b65a69 
>   ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 5ef07d0 
>   ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 0b5f3b8 
>   
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog240Test.java
>  670200c 
> 
> Diff: https://reviews.apache.org/r/48258/diff/
> 
> 
> Testing
> ---
> 
> Manually tested new cluster and upgrade.
> 
> 
> Thanks,
> 
> Robert Levas
> 
>

Re: Review Request 48290: Change HAWQ DFS Url from hawq_default to hawq_data

[Alexander Denissov]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48290/#review136358
---


Ship it!




Ship It!

- Alexander Denissov


On June 6, 2016, 7:17 p.m., Matt wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48290/
> ---
> 
> (Updated June 6, 2016, 7:17 p.m.)
> 
> 
> Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, and Lav 
> Jain.
> 
> 
> Bugs: AMBARI-17070
> https://issues.apache.org/jira/browse/AMBARI-17070
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Change HAWQ DFS Url from hawq_default to hawq_data
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-site.xml
>  3ac4e89 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/params.py
>  a8a4487 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
>  ec04fd2 
>   ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqmaster.py 70d7ed1 
>   ambari-server/src/test/python/stacks/2.3/configs/hawq_default.json b839de6 
>   ambari-web/app/data/HDP2/ha_properties.js 18c5a68 
>   
> ambari-web/test/controllers/main/admin/highAvailability/nameNode/step3_controller_test.js
>  ca55a32 
>   ambari-web/test/utils/configs/config_initializer_test.js 48eae23 
> 
> Diff: https://reviews.apache.org/r/48290/diff/
> 
> 
> Testing
> ---
> 
> Waiting for results from Hadoop QA
> 
> 
> Thanks,
> 
> Matt
> 
>

Re: Review Request 48234: Falcon server fails to start, HDP 2.4 to use data-mirroring directory, HDP 2.5 to use extensions

[Alejandro Fernandez]

> On June 3, 2016, 11:34 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon_server.py,
> >  line 94
> > 
> >
> > How could upgrade_type be None in "pre_upgrade"restart" ?

upgrade_type defaults to None if the caller doesn't specify it. This is done by 
script.py
I prefer robust coding


- Alejandro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48234/#review136136
---


On June 6, 2016, 8:04 p.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48234/
> ---
> 
> (Updated June 6, 2016, 8:04 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Di Li, Dmitro Lisnichenko, 
> Jonathan Hurley, Jayush Luniya, Nate Cole, and Tim Thorpe.
> 
> 
> Bugs: AMBARI-17037
> https://issues.apache.org/jira/browse/AMBARI-17037
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Fresh installation and EU/RU of Falcon is failing.
> In HDP 2.4, Falcon used the data mirroring folder.
> In HDP 2.5, data mirroring is no longer used and instead uses the extensions 
> folder that must be uploaded to HDFS.
> 
> This means that EU/RU from HDP 2.4 to 2.5 must also upload the extensions 
> folder to HDFS
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/constants.py
>  34d1a1a 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py
>  4c13aaa 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon.py
>  5e25325 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon_server.py
>  7c2cdf4 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/params_linux.py
>  8c2ad8e 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_features.json
>  8b669e8 
>   ambari-server/src/test/python/stacks/2.0.6/configs/default.json 8d04e36 
> 
> Diff: https://reviews.apache.org/r/48234/diff/
> 
> 
> Testing
> ---
> 
> Python unit tests passed.
> 
> --
> Total run:1052
> Total errors:0
> Total failures:0
> OK
> 
> Express Upgrade is currently broken, so will commit this patch and retest 
> once it is stable again in Hadoop Core.
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Re: Review Request 48287: RU failed because of old service was not stopped

[Jonathan Hurley]

> On June 6, 2016, 2:04 p.m., Jonathan Hurley wrote:
> > ambari-common/src/main/python/resource_management/libraries/script/script.py,
> >  lines 664-666
> > 
> >
> > I know this isn't a part of your change - but can this TODO be removed 
> > now? I'm hoping that Ranger has service checks in place. And this would 
> > alter your if-statement if removed.
> 
> Dmytro Grinenko wrote:
> i can remove TODO but not the logic, seems even simple restart in normal 
> cluster mode will fail without this. I have seen open issue in ranger team 
> for fixing this part.

If you know for a fact that Ranger doesn't have a status function yet, then 
leave it. I just figured it was something leftover from a long time ago that's 
been fixed for a while.


- Jonathan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48287/#review136331
---


On June 6, 2016, 1:20 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48287/
> ---
> 
> (Updated June 6, 2016, 1:20 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley and Nate Cole.
> 
> 
> Bugs: AMBARI-17068
> https://issues.apache.org/jira/browse/AMBARI-17068
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> STR:
> - deploy latest Ambari 2.4
> - install HDP 2.3 (HIVE and dependencies)
> - update to HDP 2.5
> 
> 
> Error message:
> 
> {code}
> Traceback (most recent call last):
> File 
> "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server.py",
>  line 211, in 
> HiveServer().execute()
> File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 257, in execute
> method(env)
> File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 669, in restart
> raise Fail("Stop command finished but process keep running.")
> resource_management.core.exceptions.Fail: Stop command finished but process 
> keep running.
> {code}
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/test/python/resource_management/TestScript.py adb8501 
>   
> ambari-common/src/main/python/resource_management/libraries/script/script.py 
> 61759cc 
> 
> Diff: https://reviews.apache.org/r/48287/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Re: Review Request 48287: RU failed because of old service was not stopped

[Nate Cole]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48287/#review136347
---


Ship it!




Ship It!

- Nate Cole


On June 6, 2016, 1:20 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48287/
> ---
> 
> (Updated June 6, 2016, 1:20 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley and Nate Cole.
> 
> 
> Bugs: AMBARI-17068
> https://issues.apache.org/jira/browse/AMBARI-17068
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> STR:
> - deploy latest Ambari 2.4
> - install HDP 2.3 (HIVE and dependencies)
> - update to HDP 2.5
> 
> 
> Error message:
> 
> {code}
> Traceback (most recent call last):
> File 
> "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server.py",
>  line 211, in 
> HiveServer().execute()
> File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 257, in execute
> method(env)
> File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 669, in restart
> raise Fail("Stop command finished but process keep running.")
> resource_management.core.exceptions.Fail: Stop command finished but process 
> keep running.
> {code}
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/test/python/resource_management/TestScript.py adb8501 
>   
> ambari-common/src/main/python/resource_management/libraries/script/script.py 
> 61759cc 
> 
> Diff: https://reviews.apache.org/r/48287/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Review Request 48290: Change HAWQ DFS Url from hawq_default to hawq_data

[Matt]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48290/
---

Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, and Lav Jain.


Bugs: AMBARI-17070
https://issues.apache.org/jira/browse/AMBARI-17070


Repository: ambari


Description
---

Change HAWQ DFS Url from hawq_default to hawq_data


Diffs
-

  
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-site.xml
 3ac4e89 
  
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/params.py
 a8a4487 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
 ec04fd2 
  ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqmaster.py 70d7ed1 
  ambari-server/src/test/python/stacks/2.3/configs/hawq_default.json b839de6 
  ambari-web/app/data/HDP2/ha_properties.js 18c5a68 
  
ambari-web/test/controllers/main/admin/highAvailability/nameNode/step3_controller_test.js
 ca55a32 
  ambari-web/test/utils/configs/config_initializer_test.js 48eae23 

Diff: https://reviews.apache.org/r/48290/diff/


Testing
---

Waiting for results from Hadoop QA


Thanks,

Matt

Re: Review Request 48229: Refactor service_advisor apis to remove passing of stack_advisor

[Tim Thorpe]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48229/#review136343
---


Ship it!




Ship It!

- Tim Thorpe


On June 6, 2016, 6:40 p.m., Lav Jain wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48229/
> ---
> 
> (Updated June 6, 2016, 6:40 p.m.)
> 
> 
> Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, Jayush 
> Luniya, Matt, and Tim Thorpe.
> 
> 
> Bugs: AMBARI-16663
> https://issues.apache.org/jira/browse/AMBARI-16663
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> The original intent was to move the common functions that are used by both 
> stack and service advisors to a separate utils file to avoid passing 
> stack_advisor class into service advisor apis.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py
>  28eb82f 
>   
> ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py 
> 9b34171 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
> 1dedf3d 
>   ambari-server/src/main/resources/stacks/service_advisor.py 3d6c293 
>   ambari-server/src/main/resources/stacks/stack_advisor.py beba225 
>   ambari-server/src/test/python/stacks/2.3/HAWQ/test_service_advisor.py 
> 5750938 
>   ambari-server/src/test/python/stacks/2.3/PXF/test_service_advisor.py 
> c3a63cc 
>   ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 
> 424a386 
> 
> Diff: https://reviews.apache.org/r/48229/diff/
> 
> 
> Testing
> ---
> 
> Lavs-MacBook-Pro:common ljain$ python -m discover -v
> test_createComponentLayoutRecommendations_hawq_1_Host 
> (test_stack_advisor.TestHDP23StackAdvisor) ... ServiceAdvisor implementation 
> for service HAWQ was loaded
> [u'c6401.ambari.apache.org']
> [u'c6401.ambari.apache.org']
> ok
> test_createComponentLayoutRecommendations_hawq_3_Hosts 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test that HAWQSTANDBY is recommended on a 3-node cluster ... ServiceAdvisor 
> implementation for service HAWQ was loaded
> [u'c6401.ambari.apache.org', u'c6402.ambari.apache.org', 
> u'c6403.ambari.apache.org']
> [u'c6401.ambari.apache.org', u'c6402.ambari.apache.org', 
> u'c6403.ambari.apache.org']
> ok
> test_createComponentLayoutRecommendations_hawqsegment_add_service_wizard_already_installed
>  (test_stack_advisor.TestHDP23StackAdvisor)
> Test that HAWQSEGMENT does not get recommended during Add Service Wizard, 
> when HAWQ has already been installed ... ServiceAdvisor implementation for 
> service HAWQ was loaded
> ok
> test_createComponentLayoutRecommendations_hawqsegment_add_service_wizard_to_be_installed
>  (test_stack_advisor.TestHDP23StackAdvisor)
> Test that HAWQSEGMENT gets recommended correctly during Add Service Wizard, 
> when HAWQ is selected for installation ... ServiceAdvisor implementation for 
> service HAWQ was loaded
> ok
> test_createComponentLayoutRecommendations_hawqsegment_cluster_install 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test that HAWQSEGMENT gets recommended correctly during Cluster Install 
> Wizard, when HAWQ is selected for installation ... ServiceAdvisor 
> implementation for service HAWQ was loaded
> ok
> test_createComponentLayoutRecommendations_no_hawq_3_Hosts 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test no failures when there are no HAWQ components ... ok
> test_createComponentLayoutRecommendations_pxf_add_service_wizard_already_installed
>  (test_stack_advisor.TestHDP23StackAdvisor)
> Test that PXF does not get recommended during Add Service Wizard, when PXF 
> has already been installed ... ServiceAdvisor implementation for service PXF 
> was loaded
> ok
> test_createComponentLayoutRecommendations_pxf_add_service_wizard_to_be_installed
>  (test_stack_advisor.TestHDP23StackAdvisor)
> Test that PXF gets recommended correctly during Add Service Wizard, when PXF 
> is selected for installation ... ServiceAdvisor implementation for service 
> PXF was loaded
> ok
> test_createComponentLayoutRecommendations_pxf_cluster_install 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test that PXF gets recommended correctly during Cluster Install Wizard, when 
> PXF is selected for installation ... ServiceAdvisor implementation for 
> service PXF was loaded
> ok
> test_getComponentLayoutValidations_hawq_3_Hosts 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test layout validations for HAWQ components on a 3-node cluster ... 
> ServiceAdvisor implementation for service HAWQ was loaded
> ok
> test_getComponentLayoutValidations_hawqsegment_not_co_located_with_datanode 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test validation warning for HAWQ segment not colocated with DATANODE 

Re: Review Request 48229: Refactor service_advisor apis to remove passing of stack_advisor

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48229/#review136342
---


Ship it!




Ship It!

- Alejandro Fernandez


On June 6, 2016, 6:40 p.m., Lav Jain wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48229/
> ---
> 
> (Updated June 6, 2016, 6:40 p.m.)
> 
> 
> Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, Jayush 
> Luniya, Matt, and Tim Thorpe.
> 
> 
> Bugs: AMBARI-16663
> https://issues.apache.org/jira/browse/AMBARI-16663
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> The original intent was to move the common functions that are used by both 
> stack and service advisors to a separate utils file to avoid passing 
> stack_advisor class into service advisor apis.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py
>  28eb82f 
>   
> ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py 
> 9b34171 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
> 1dedf3d 
>   ambari-server/src/main/resources/stacks/service_advisor.py 3d6c293 
>   ambari-server/src/main/resources/stacks/stack_advisor.py beba225 
>   ambari-server/src/test/python/stacks/2.3/HAWQ/test_service_advisor.py 
> 5750938 
>   ambari-server/src/test/python/stacks/2.3/PXF/test_service_advisor.py 
> c3a63cc 
>   ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 
> 424a386 
> 
> Diff: https://reviews.apache.org/r/48229/diff/
> 
> 
> Testing
> ---
> 
> Lavs-MacBook-Pro:common ljain$ python -m discover -v
> test_createComponentLayoutRecommendations_hawq_1_Host 
> (test_stack_advisor.TestHDP23StackAdvisor) ... ServiceAdvisor implementation 
> for service HAWQ was loaded
> [u'c6401.ambari.apache.org']
> [u'c6401.ambari.apache.org']
> ok
> test_createComponentLayoutRecommendations_hawq_3_Hosts 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test that HAWQSTANDBY is recommended on a 3-node cluster ... ServiceAdvisor 
> implementation for service HAWQ was loaded
> [u'c6401.ambari.apache.org', u'c6402.ambari.apache.org', 
> u'c6403.ambari.apache.org']
> [u'c6401.ambari.apache.org', u'c6402.ambari.apache.org', 
> u'c6403.ambari.apache.org']
> ok
> test_createComponentLayoutRecommendations_hawqsegment_add_service_wizard_already_installed
>  (test_stack_advisor.TestHDP23StackAdvisor)
> Test that HAWQSEGMENT does not get recommended during Add Service Wizard, 
> when HAWQ has already been installed ... ServiceAdvisor implementation for 
> service HAWQ was loaded
> ok
> test_createComponentLayoutRecommendations_hawqsegment_add_service_wizard_to_be_installed
>  (test_stack_advisor.TestHDP23StackAdvisor)
> Test that HAWQSEGMENT gets recommended correctly during Add Service Wizard, 
> when HAWQ is selected for installation ... ServiceAdvisor implementation for 
> service HAWQ was loaded
> ok
> test_createComponentLayoutRecommendations_hawqsegment_cluster_install 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test that HAWQSEGMENT gets recommended correctly during Cluster Install 
> Wizard, when HAWQ is selected for installation ... ServiceAdvisor 
> implementation for service HAWQ was loaded
> ok
> test_createComponentLayoutRecommendations_no_hawq_3_Hosts 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test no failures when there are no HAWQ components ... ok
> test_createComponentLayoutRecommendations_pxf_add_service_wizard_already_installed
>  (test_stack_advisor.TestHDP23StackAdvisor)
> Test that PXF does not get recommended during Add Service Wizard, when PXF 
> has already been installed ... ServiceAdvisor implementation for service PXF 
> was loaded
> ok
> test_createComponentLayoutRecommendations_pxf_add_service_wizard_to_be_installed
>  (test_stack_advisor.TestHDP23StackAdvisor)
> Test that PXF gets recommended correctly during Add Service Wizard, when PXF 
> is selected for installation ... ServiceAdvisor implementation for service 
> PXF was loaded
> ok
> test_createComponentLayoutRecommendations_pxf_cluster_install 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test that PXF gets recommended correctly during Cluster Install Wizard, when 
> PXF is selected for installation ... ServiceAdvisor implementation for 
> service PXF was loaded
> ok
> test_getComponentLayoutValidations_hawq_3_Hosts 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test layout validations for HAWQ components on a 3-node cluster ... 
> ServiceAdvisor implementation for service HAWQ was loaded
> ok
> test_getComponentLayoutValidations_hawqsegment_not_co_located_with_datanode 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test validation warning for HAWQ segment not colocated with 

Re: Review Request 48242: Takes long time to start or fail to start service after enabling SSL due to "dfs.https.enable"

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48242/#review136341
---


Ship it!




Thank you for making this change Richard

- Alejandro Fernandez


On June 3, 2016, 11:56 p.m., Richard Zang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48242/
> ---
> 
> (Updated June 3, 2016, 11:56 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Andrew Onischuk, and Sumit 
> Mohanty.
> 
> 
> Bugs: AMBARI-17039
> https://issues.apache.org/jira/browse/AMBARI-17039
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Check both dfs.http.policy and deprecated dfs.https.enable
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/hdfs_utils.py
>  PRE-CREATION 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/namenode_ha_utils.py
>  919ccb5 
>   
> ambari-common/src/main/python/resource_management/libraries/providers/hdfs_resource.py
>  de46c1c 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py
>  b634ce9 
> 
> Diff: https://reviews.apache.org/r/48242/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Richard Zang
> 
>

Re: Review Request 48289: AMBARI-10908 Usability: ability to perform bulk delete host

[Zhe (Joe) Wang]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48289/
---

(Updated June 6, 2016, 6:22 p.m.)


Review request for Ambari, Ajit Kumar, Jaimin Jetly, Richard Zang, Xi Wang, and 
Yusaku Sako.


Bugs: AMBARI-10908
https://issues.apache.org/jira/browse/AMBARI-10908


Repository: ambari


Description
---

On Hosts page, provide ability to delete hosts in bulk.


Diffs (updated)
-

  ambari-web/app/controllers/main/host/bulk_operations_controller.js 09d6a52 
  ambari-web/app/messages.js e8ca9ae 
  ambari-web/app/styles/application.less 5ee09cc 
  ambari-web/app/templates/main/host/delete_hosts_dry_run_popup.hbs 
PRE-CREATION 
  ambari-web/app/templates/main/host/delete_hosts_popup.hbs PRE-CREATION 
  ambari-web/app/templates/main/host/delete_hosts_result_popup.hbs PRE-CREATION 
  ambari-web/app/utils/ajax/ajax.js 195dab4 
  ambari-web/app/views/main/host/hosts_table_menu_view.js 5e4b0e0 

Diff: https://reviews.apache.org/r/48289/diff/


Testing
---

Local ambari-web test passed.
28648 tests complete (24 seconds)
154 tests pending
Manual testing done.


Thanks,

Zhe (Joe) Wang

Re: Review Request 48044: Provide context for hdp-select failures during ambari component install

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48044/#review136336
---


Fix it, then Ship it!





ambari-common/src/main/python/resource_management/libraries/functions/hdp_select.py
 (line 136)


Please add a comment as to why this was done.


- Alejandro Fernandez


On June 6, 2016, 3:15 p.m., Laszlo Puskas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48044/
> ---
> 
> (Updated June 6, 2016, 3:15 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Oliver Szabo, Sumit Mohanty, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-16952
> https://issues.apache.org/jira/browse/AMBARI-16952
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In case the hdp-select command fails during a service / component 
> installation there's no contextual information about the cause of the failure.
> This issue is for logging information about the machine on which the 
> hdp-select command fails.
> This solution wraps hdp-select command calls in a try/catch block and logs 
> failure / hdp installationrelated information.
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/hdp_select.py
>  9a3201e 
> 
> Diff: https://reviews.apache.org/r/48044/diff/
> 
> 
> Testing
> ---
> 
> Unit tests passed.
> Manual testing underway please don't commit yet.
> // found some major differences between the branches / investigating it
> 
> 
> Thanks,
> 
> Laszlo Puskas
> 
>

Re: Review Request 48287: RU failed because of old service was not stopped

[Dmytro Grinenko]

> On June 6, 2016, 6:04 p.m., Jonathan Hurley wrote:
> > ambari-common/src/main/python/resource_management/libraries/script/script.py,
> >  lines 664-666
> > 
> >
> > I know this isn't a part of your change - but can this TODO be removed 
> > now? I'm hoping that Ranger has service checks in place. And this would 
> > alter your if-statement if removed.

i can remove TODO but not the logic, seems even simple restart in normal 
cluster mode will fail without this. I have seen open issue in ranger team for 
fixing this part.


- Dmytro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48287/#review136331
---


On June 6, 2016, 5:20 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48287/
> ---
> 
> (Updated June 6, 2016, 5:20 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley and Nate Cole.
> 
> 
> Bugs: AMBARI-17068
> https://issues.apache.org/jira/browse/AMBARI-17068
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> STR:
> - deploy latest Ambari 2.4
> - install HDP 2.3 (HIVE and dependencies)
> - update to HDP 2.5
> 
> 
> Error message:
> 
> {code}
> Traceback (most recent call last):
> File 
> "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server.py",
>  line 211, in 
> HiveServer().execute()
> File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 257, in execute
> method(env)
> File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 669, in restart
> raise Fail("Stop command finished but process keep running.")
> resource_management.core.exceptions.Fail: Stop command finished but process 
> keep running.
> {code}
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/test/python/resource_management/TestScript.py adb8501 
>   
> ambari-common/src/main/python/resource_management/libraries/script/script.py 
> 61759cc 
> 
> Diff: https://reviews.apache.org/r/48287/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Re: Review Request 48288: Microsoft-r install script should use Ambari libraries instead of direct shell calls

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48288/#review136333
---


Fix it, then Ship it!





contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/package/scripts/microsoft_r.py
 (line 30)


Ambari QA user should not be hardcoded since it can be changed during 
installation.


- Alejandro Fernandez


On June 6, 2016, 5:30 p.m., Balázs Bence Sári wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48288/
> ---
> 
> (Updated June 6, 2016, 5:30 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jayush Luniya, Srimanth 
> Gunturi, Sumit Gupta, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-17067
> https://issues.apache.org/jira/browse/AMBARI-17067
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Made the following changes:
> - Use Ambari Python API only
> - Use new rpm's
> - Cleaned up imports.
> - Added supported HDP versions to the mpack
> 
> 
> Diffs
> -
> 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/package/scripts/microsoft_r.py
>  5b6e482 
>   contrib/management-packs/microsoft-r_mpack/src/main/resources/mpack.json 
> 17c2f54 
> 
> Diff: https://reviews.apache.org/r/48288/diff/
> 
> 
> Testing
> ---
> 
> On a a single node cluster (CentOS 6.4, Ambari 2.2 upgraded to 2.4 snapshot 
> build, HDP 2.3), did the following testing:
> - Built the management pack with mvn install
> - Installed it on the Ambari Server by running this command: ambari-server 
> install-mpack --mpack microsoft-r_mpack-1.0.0.0-SNAPSHOT.tar.gz
> - Restarted the Ambari Server
> - On the UI, chose add service and selected Microsoft R.
> - Install completed without errors. It had a warning due to missing service 
> check script (covered by another Jira ticket).
> 
> 
> Thanks,
> 
> Balázs Bence Sári
> 
>

Re: Review Request 48287: RU failed because of old service was not stopped

[Dmytro Grinenko]

> On June 6, 2016, 5:31 p.m., Nate Cole wrote:
> > I only see two changes here, and it's just logging.  Was there some missed 
> > files?
> 
> Jonathan Hurley wrote:
> Same - only 2 changes and they are for logging.
> 
> Dmytro Grinenko wrote:
> no, as you can see, self.status(env)  would be skipped and this is goal 
> of the patch
> 
> Jonathan Hurley wrote:
> Ah, I see. So, this change skips status checks during upgrade?
> 
> Nate Cole wrote:
> Hmm.  We do allow service checks if the upgrade is suspended.  Should 
> that be restricted as well?

yep, this would be perfect solution for 2.4.0 coz upgrade behaviour would be 
consistent with previous releases. While for next releases we can add to *env 
properties some kind of perfect tunning, where to allow this feature. But, 
IMHO, this is dangerous thing during RU.


- Dmytro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48287/#review136319
---


On June 6, 2016, 5:20 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48287/
> ---
> 
> (Updated June 6, 2016, 5:20 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley and Nate Cole.
> 
> 
> Bugs: AMBARI-17068
> https://issues.apache.org/jira/browse/AMBARI-17068
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> STR:
> - deploy latest Ambari 2.4
> - install HDP 2.3 (HIVE and dependencies)
> - update to HDP 2.5
> 
> 
> Error message:
> 
> {code}
> Traceback (most recent call last):
> File 
> "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server.py",
>  line 211, in 
> HiveServer().execute()
> File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 257, in execute
> method(env)
> File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 669, in restart
> raise Fail("Stop command finished but process keep running.")
> resource_management.core.exceptions.Fail: Stop command finished but process 
> keep running.
> {code}
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/test/python/resource_management/TestScript.py adb8501 
>   
> ambari-common/src/main/python/resource_management/libraries/script/script.py 
> 61759cc 
> 
> Diff: https://reviews.apache.org/r/48287/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Re: Review Request 48287: RU failed because of old service was not stopped

[Jonathan Hurley]

> On June 6, 2016, 1:31 p.m., Nate Cole wrote:
> > I only see two changes here, and it's just logging.  Was there some missed 
> > files?
> 
> Jonathan Hurley wrote:
> Same - only 2 changes and they are for logging.
> 
> Dmytro Grinenko wrote:
> no, as you can see, self.status(env)  would be skipped and this is goal 
> of the patch

Ah, I see. So, this change skips status checks during upgrade?


- Jonathan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48287/#review136319
---


On June 6, 2016, 1:20 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48287/
> ---
> 
> (Updated June 6, 2016, 1:20 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley and Nate Cole.
> 
> 
> Bugs: AMBARI-17068
> https://issues.apache.org/jira/browse/AMBARI-17068
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> STR:
> - deploy latest Ambari 2.4
> - install HDP 2.3 (HIVE and dependencies)
> - update to HDP 2.5
> 
> 
> Error message:
> 
> {code}
> Traceback (most recent call last):
> File 
> "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server.py",
>  line 211, in 
> HiveServer().execute()
> File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 257, in execute
> method(env)
> File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 669, in restart
> raise Fail("Stop command finished but process keep running.")
> resource_management.core.exceptions.Fail: Stop command finished but process 
> keep running.
> {code}
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/test/python/resource_management/TestScript.py adb8501 
>   
> ambari-common/src/main/python/resource_management/libraries/script/script.py 
> 61759cc 
> 
> Diff: https://reviews.apache.org/r/48287/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Review Request 48289: AMBARI-10908 Usability: ability to perform bulk delete host

[Zhe (Joe) Wang]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48289/
---

Review request for Ambari, Ajit Kumar, Jaimin Jetly, Richard Zang, Xi Wang, and 
Yusaku Sako.


Bugs: AMBARI-10908
https://issues.apache.org/jira/browse/AMBARI-10908


Repository: ambari


Description
---

On Hosts page, provide ability to delete hosts in bulk.


Diffs
-

  ambari-web/app/controllers/main/host/bulk_operations_controller.js 09d6a52 
  ambari-web/app/messages.js ffc6eb2 
  ambari-web/app/styles/application.less 5ee09cc 
  ambari-web/app/templates/main/host/delete_hosts_dry_run_popup.hbs 
PRE-CREATION 
  ambari-web/app/templates/main/host/delete_hosts_popup.hbs PRE-CREATION 
  ambari-web/app/templates/main/host/delete_hosts_result_popup.hbs PRE-CREATION 
  ambari-web/app/utils/ajax/ajax.js 195dab4 
  ambari-web/app/views/main/host/hosts_table_menu_view.js 5e4b0e0 

Diff: https://reviews.apache.org/r/48289/diff/


Testing
---

Local ambari-web test passed.
28648 tests complete (24 seconds)
154 tests pending
Manual testing done.


Thanks,

Zhe (Joe) Wang

Re: Review Request 48287: RU failed because of old service was not stopped

[Jonathan Hurley]

> On June 6, 2016, 1:31 p.m., Nate Cole wrote:
> > I only see two changes here, and it's just logging.  Was there some missed 
> > files?

Same - only 2 changes and they are for logging.


- Jonathan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48287/#review136319
---


On June 6, 2016, 1:20 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48287/
> ---
> 
> (Updated June 6, 2016, 1:20 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley and Nate Cole.
> 
> 
> Bugs: AMBARI-17068
> https://issues.apache.org/jira/browse/AMBARI-17068
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> STR:
> - deploy latest Ambari 2.4
> - install HDP 2.3 (HIVE and dependencies)
> - update to HDP 2.5
> 
> 
> Error message:
> 
> {code}
> Traceback (most recent call last):
> File 
> "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server.py",
>  line 211, in 
> HiveServer().execute()
> File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 257, in execute
> method(env)
> File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 669, in restart
> raise Fail("Stop command finished but process keep running.")
> resource_management.core.exceptions.Fail: Stop command finished but process 
> keep running.
> {code}
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/test/python/resource_management/TestScript.py adb8501 
>   
> ambari-common/src/main/python/resource_management/libraries/script/script.py 
> 61759cc 
> 
> Diff: https://reviews.apache.org/r/48287/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Re: Review Request 48273: AMBARI-17054 : Configure Atlas Ranger Plugin

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48273/#review136305
---




ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata_server.py
 (lines 74 - 78)


You're uploading things to HDFS on Ranger start; but this will fail on any 
stack upgrade as HDFS is started after Ranger.



ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py
 (line 241)


Rename to `retry_enabled` to match other propery naming conventions.



ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/setup_ranger_atlas.py
 (lines 31 - 33)


Doesn't Ranger start before HDFS?


- Jonathan Hurley


On June 6, 2016, 11:26 a.m., Gautam Borad wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48273/
> ---
> 
> (Updated June 6, 2016, 11:26 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Srimanth 
> Gunturi, and Velmurugan Periasamy.
> 
> 
> Bugs: AMBARI-17054
> https://issues.apache.org/jira/browse/AMBARI-17054
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Patch contains changes to support Ranger Atlas Plugin from Ambari.
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/constants.py
>  34d1a1a 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata_server.py
>  1e9e7a7 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py
>  a79a456 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/setup_ranger_atlas.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/status_params.py
>  4c54214 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-env.xml
>  4db7f45 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.6.0/themes/theme_version_3.json
>  0f7b0c0 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_features.json
>  8b669e8 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/configuration/application-properties.xml
>  98cbc30 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/configuration/ranger-atlas-audit.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/configuration/ranger-atlas-plugin-properties.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/configuration/ranger-atlas-policymgr-ssl.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/configuration/ranger-atlas-security.xml
>  PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
> 379c268 
> 
> Diff: https://reviews.apache.org/r/48273/diff/
> 
> 
> Testing
> ---
> 
> Verified : 
> 
> 1) Enable / disable actions for Ranger Atlas Plugin
> 2) Checked Ranger plugin comminication. 
> 3) Checked policy enforecements on Atlas based on Ranger's policies.
> 4) Verified stack advisor recommendations to chnange following properties : 
> 
> atlas.authorizer.impl=org.apache.ranger.authorization.atlas.authorizer.RangerAtlasAuthorizer
> ranger-atlas-plugin-enabled=yes
> 5) Verified Ranger service creation (while enabling Ranger Atlas Plugin)
> 
> 
> Thanks,
> 
> Gautam Borad
> 
>

Re: Review Request 48287: RU failed because of old service was not stopped

[Nate Cole]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48287/#review136319
---



I only see two changes here, and it's just logging.  Was there some missed 
files?

- Nate Cole


On June 6, 2016, 1:20 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48287/
> ---
> 
> (Updated June 6, 2016, 1:20 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley and Nate Cole.
> 
> 
> Bugs: AMBARI-17068
> https://issues.apache.org/jira/browse/AMBARI-17068
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> STR:
> - deploy latest Ambari 2.4
> - install HDP 2.3 (HIVE and dependencies)
> - update to HDP 2.5
> 
> 
> Error message:
> 
> {code}
> Traceback (most recent call last):
> File 
> "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server.py",
>  line 211, in 
> HiveServer().execute()
> File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 257, in execute
> method(env)
> File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 669, in restart
> raise Fail("Stop command finished but process keep running.")
> resource_management.core.exceptions.Fail: Stop command finished but process 
> keep running.
> {code}
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/test/python/resource_management/TestScript.py adb8501 
>   
> ambari-common/src/main/python/resource_management/libraries/script/script.py 
> 61759cc 
> 
> Diff: https://reviews.apache.org/r/48287/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Re: Review Request 48288: Microsoft-r install script should use Ambari libraries instead of direct shell calls

[Balázs Bence Sári]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48288/
---

(Updated June 6, 2016, 5:30 p.m.)


Review request for Ambari, Alejandro Fernandez, Jayush Luniya, Srimanth 
Gunturi, Sumit Gupta, and Yusaku Sako.


Bugs: AMBARI-17067
https://issues.apache.org/jira/browse/AMBARI-17067


Repository: ambari


Description (updated)
---

Made the following changes:
- Use Ambari Python API only
- Use new rpm's
- Cleaned up imports.
- Added supported HDP versions to the mpack


Diffs
-

  
contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/package/scripts/microsoft_r.py
 5b6e482 
  contrib/management-packs/microsoft-r_mpack/src/main/resources/mpack.json 
17c2f54 

Diff: https://reviews.apache.org/r/48288/diff/


Testing
---

On a a single node cluster (CentOS 6.4, Ambari 2.2 upgraded to 2.4 snapshot 
build, HDP 2.3), did the following testing:
- Built the management pack with mvn install
- Installed it on the Ambari Server by running this command: ambari-server 
install-mpack --mpack microsoft-r_mpack-1.0.0.0-SNAPSHOT.tar.gz
- Restarted the Ambari Server
- On the UI, chose add service and selected Microsoft R.
- Install completed without errors. It had a warning due to missing service 
check script (covered by another Jira ticket).


Thanks,

Balázs Bence Sári

Re: Review Request 48229: Refactor service_advisor apis to remove passing of stack_advisor

[Lav Jain]

> On June 6, 2016, 12:43 p.m., Tim Thorpe wrote:
> > ambari-server/src/main/resources/stacks/service_advisor.py, line 67
> > 
> >
> > I originally had implemented the service advisor like this.  I had 
> > modified it specifically NOT to inherit from the DefaultStackAdvisor after 
> > comments suggesting that the inheritance didn't make sense.
> > 
> > The way I see it both the stack advisor and service advisor should 
> > inherit the core functions from a generic advisor but this implementation 
> > is the easiest.

There seems to be a lot of duplicate code and this effort is streamlining it 
with minimal intrusion to the stack advisor.


- Lav


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48229/#review136240
---


On June 5, 2016, 10:18 p.m., Lav Jain wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48229/
> ---
> 
> (Updated June 5, 2016, 10:18 p.m.)
> 
> 
> Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, Jayush 
> Luniya, Matt, and Tim Thorpe.
> 
> 
> Bugs: AMBARI-16663
> https://issues.apache.org/jira/browse/AMBARI-16663
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> The original intent was to move the common functions that are used by both 
> stack and service advisors to a separate utils file to avoid passing 
> stack_advisor class into service advisor apis.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py
>  28eb82f 
>   
> ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py 
> 9b34171 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
> 1dedf3d 
>   ambari-server/src/main/resources/stacks/service_advisor.py 3d6c293 
>   ambari-server/src/main/resources/stacks/stack_advisor.py beba225 
>   ambari-server/src/test/python/stacks/2.3/HAWQ/test_service_advisor.py 
> 5750938 
>   ambari-server/src/test/python/stacks/2.3/PXF/test_service_advisor.py 
> c3a63cc 
>   ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 
> 424a386 
> 
> Diff: https://reviews.apache.org/r/48229/diff/
> 
> 
> Testing
> ---
> 
> Lavs-MacBook-Pro:common ljain$ python -m discover -v
> test_createComponentLayoutRecommendations_hawq_1_Host 
> (test_stack_advisor.TestHDP23StackAdvisor) ... ServiceAdvisor implementation 
> for service HAWQ was loaded
> [u'c6401.ambari.apache.org']
> [u'c6401.ambari.apache.org']
> ok
> test_createComponentLayoutRecommendations_hawq_3_Hosts 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test that HAWQSTANDBY is recommended on a 3-node cluster ... ServiceAdvisor 
> implementation for service HAWQ was loaded
> [u'c6401.ambari.apache.org', u'c6402.ambari.apache.org', 
> u'c6403.ambari.apache.org']
> [u'c6401.ambari.apache.org', u'c6402.ambari.apache.org', 
> u'c6403.ambari.apache.org']
> ok
> test_createComponentLayoutRecommendations_hawqsegment_add_service_wizard_already_installed
>  (test_stack_advisor.TestHDP23StackAdvisor)
> Test that HAWQSEGMENT does not get recommended during Add Service Wizard, 
> when HAWQ has already been installed ... ServiceAdvisor implementation for 
> service HAWQ was loaded
> ok
> test_createComponentLayoutRecommendations_hawqsegment_add_service_wizard_to_be_installed
>  (test_stack_advisor.TestHDP23StackAdvisor)
> Test that HAWQSEGMENT gets recommended correctly during Add Service Wizard, 
> when HAWQ is selected for installation ... ServiceAdvisor implementation for 
> service HAWQ was loaded
> ok
> test_createComponentLayoutRecommendations_hawqsegment_cluster_install 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test that HAWQSEGMENT gets recommended correctly during Cluster Install 
> Wizard, when HAWQ is selected for installation ... ServiceAdvisor 
> implementation for service HAWQ was loaded
> ok
> test_createComponentLayoutRecommendations_no_hawq_3_Hosts 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test no failures when there are no HAWQ components ... ok
> test_createComponentLayoutRecommendations_pxf_add_service_wizard_already_installed
>  (test_stack_advisor.TestHDP23StackAdvisor)
> Test that PXF does not get recommended during Add Service Wizard, when PXF 
> has already been installed ... ServiceAdvisor implementation for service PXF 
> was loaded
> ok
> test_createComponentLayoutRecommendations_pxf_add_service_wizard_to_be_installed
>  (test_stack_advisor.TestHDP23StackAdvisor)
> Test that PXF gets recommended correctly during Add Service Wizard, when PXF 
> is selected for installation ... ServiceAdvisor implementation for service 
> PXF was loaded
> ok
> 

Review Request 48288: Microsoft-r install script should use Ambari libraries instead of direct shell calls

[Balázs Bence Sári]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48288/
---

Review request for Ambari, Alejandro Fernandez, Jayush Luniya, Srimanth 
Gunturi, Sumit Gupta, and Yusaku Sako.


Bugs: AMBARI-17067
https://issues.apache.org/jira/browse/AMBARI-17067


Repository: ambari


Description
---

Made the following changes:
- Use Ambari Python API only
- Use new rpm's
- Cleaned up imports.


Diffs
-

  
contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/package/scripts/microsoft_r.py
 5b6e482 
  contrib/management-packs/microsoft-r_mpack/src/main/resources/mpack.json 
17c2f54 

Diff: https://reviews.apache.org/r/48288/diff/


Testing
---

On a a single node cluster (CentOS 6.4, Ambari 2.2 upgraded to 2.4 snapshot 
build, HDP 2.3), did the following testing:
- Built the management pack with mvn install
- Installed it on the Ambari Server by running this command: ambari-server 
install-mpack --mpack microsoft-r_mpack-1.0.0.0-SNAPSHOT.tar.gz
- Restarted the Ambari Server
- On the UI, chose add service and selected Microsoft R.
- Install completed without errors. It had a warning due to missing service 
check script (covered by another Jira ticket).


Thanks,

Balázs Bence Sári

Re: Review Request 48287: RU failed because of old service was not stopped

[Dmitro Lisnichenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48287/
---

(Updated June 6, 2016, 8:20 p.m.)


Review request for Ambari, Jonathan Hurley and Nate Cole.


Bugs: AMBARI-17068
https://issues.apache.org/jira/browse/AMBARI-17068


Repository: ambari


Description
---

STR:
- deploy latest Ambari 2.4
- install HDP 2.3 (HIVE and dependencies)
- update to HDP 2.5


Error message:

{code}
Traceback (most recent call last):
File 
"/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server.py",
 line 211, in 
HiveServer().execute()
File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
 line 257, in execute
method(env)
File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
 line 669, in restart
raise Fail("Stop command finished but process keep running.")
resource_management.core.exceptions.Fail: Stop command finished but process 
keep running.
{code}


Diffs
-

  ambari-agent/src/test/python/resource_management/TestScript.py adb8501 
  ambari-common/src/main/python/resource_management/libraries/script/script.py 
61759cc 

Diff: https://reviews.apache.org/r/48287/diff/


Testing
---

mvn clean test


Thanks,

Dmitro Lisnichenko

Re: Review Request 48262: AMBARI-16852: Update metrics collector pom.xml to use httpclient-4.2.5.jar

[Qin Liu]

> On June 6, 2016, 4:34 p.m., Aravindan Vijayan wrote:
> > Ship It!

Thank you so much Aravindan!


- Qin


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48262/#review136289
---


On June 6, 2016, 6:26 a.m., Qin Liu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48262/
> ---
> 
> (Updated June 6, 2016, 6:26 a.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Dmytro Sen, and Sid Wagle.
> 
> 
> Bugs: AMBARI-16852
> https://issues.apache.org/jira/browse/AMBARI-16852
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> httpclient-4.0.1.jar used by Ambari Metrics collector does not have method 
> URLEncodedUtils.parse (final URI uri, final String encoding). This will cause 
> the metrics not to be populated and the collector log to be flooded with the 
> following exception when the Ambari Metrics service is in distributed mode 
> and the user configures http authentication filter without enabling Kerberos:
> 
> 21:54:08,215 ERROR [1831490216@qtp-1883237069-7582] log:87 - Error for 
> /ws/v1/timeline/metrics
> java.lang.NoSuchMethodError: 
> org.apache.http.client.utils.URLEncodedUtils.parse(Ljava/lang/String;Ljava/nio/charset/Charset;)Ljava/util/List;
> at 
> org.apache.hadoop.security.authentication.server.PseudoAuthenticationHandler.getUserName(PseudoAuthenticationHandler.java:143)
> at 
> org.apache.hadoop.security.authentication.server.PseudoAuthenticationHandler.authenticate(PseudoAuthenticationHandler.java:179)
> at 
> org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:507)
> at 
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at 
> org.apache.hadoop.http.HttpServer2$QuotingInputFilter.doFilter(HttpServer2.java:1224)
> at 
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
> at 
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
> at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> 
> Steps to reproduce:
> 1. install a cluster with default from Ambari Web UI.
> 2. configure Ambari Metrics service in distributed mode.
> 3. Add the following properties in Custom core-site from Ambari Web UI:
> 1). 
> hadoop.http.authentication.signature.secret.file=/etc/security/http_secret.
> 2). 
> hadoop.http.filter.initializers=org.apache.hadoop.security.AuthenticationFilterInitialize.
> 4. create /etc/security/http_secret text file on each node.
> 5. restart HDFS, YARN, Mapreduce2, HBase, Kafka, and Ambari Metrics service.
> 6. Metrics on dashboard and HDFS, YARN, HBase, Kafka, and Amabari Metrics 
> will show No Data Available.
> 
> 
> Diffs
> -
> 
>   ambari-metrics/ambari-metrics-timelineservice/pom.xml 29f7812 
> 
> Diff: https://reviews.apache.org/r/48262/diff/
> 
> 
> Testing
> ---
> 
> FIX:
> 1. update Ambari Metrics collector pom.xml to use httpclient-4.2.5.jar.
> 2. without the fix, the collector lib contains httpclient-4.0.1.jar and 
> httpcore-4.2.5.jar.
> 3. with the fix, the collector lib will have httpclient-4.2.5.jar and 
> httpcore-4.2.4.jar.
> 
> TESTS:
> 1. The Ambari Metrics unit tests.
> 2. Also tested with HDP UI - after replacing httpclient-4.0.1.jar and 
> httpcore-4.2.5.jar with httpclient-4.2.5.jar and httpcore-4.2.4.jar, the 
> missing metrics show up.
> 
> 
> Thanks,
> 
> Qin Liu
> 
>

Re: Review Request 48262: AMBARI-16852: Update metrics collector pom.xml to use httpclient-4.2.5.jar

[Qin Liu]

> On June 6, 2016, 4:32 p.m., Dmytro Sen wrote:
> > Ship It!
> 
> Dmytro Sen wrote:
> Committed to trunk and branch-2.4.
> Please close this review board

Thank you so much Dmytro!


- Qin


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48262/#review136288
---


On June 6, 2016, 6:26 a.m., Qin Liu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48262/
> ---
> 
> (Updated June 6, 2016, 6:26 a.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Dmytro Sen, and Sid Wagle.
> 
> 
> Bugs: AMBARI-16852
> https://issues.apache.org/jira/browse/AMBARI-16852
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> httpclient-4.0.1.jar used by Ambari Metrics collector does not have method 
> URLEncodedUtils.parse (final URI uri, final String encoding). This will cause 
> the metrics not to be populated and the collector log to be flooded with the 
> following exception when the Ambari Metrics service is in distributed mode 
> and the user configures http authentication filter without enabling Kerberos:
> 
> 21:54:08,215 ERROR [1831490216@qtp-1883237069-7582] log:87 - Error for 
> /ws/v1/timeline/metrics
> java.lang.NoSuchMethodError: 
> org.apache.http.client.utils.URLEncodedUtils.parse(Ljava/lang/String;Ljava/nio/charset/Charset;)Ljava/util/List;
> at 
> org.apache.hadoop.security.authentication.server.PseudoAuthenticationHandler.getUserName(PseudoAuthenticationHandler.java:143)
> at 
> org.apache.hadoop.security.authentication.server.PseudoAuthenticationHandler.authenticate(PseudoAuthenticationHandler.java:179)
> at 
> org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:507)
> at 
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at 
> org.apache.hadoop.http.HttpServer2$QuotingInputFilter.doFilter(HttpServer2.java:1224)
> at 
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
> at 
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
> at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> 
> Steps to reproduce:
> 1. install a cluster with default from Ambari Web UI.
> 2. configure Ambari Metrics service in distributed mode.
> 3. Add the following properties in Custom core-site from Ambari Web UI:
> 1). 
> hadoop.http.authentication.signature.secret.file=/etc/security/http_secret.
> 2). 
> hadoop.http.filter.initializers=org.apache.hadoop.security.AuthenticationFilterInitialize.
> 4. create /etc/security/http_secret text file on each node.
> 5. restart HDFS, YARN, Mapreduce2, HBase, Kafka, and Ambari Metrics service.
> 6. Metrics on dashboard and HDFS, YARN, HBase, Kafka, and Amabari Metrics 
> will show No Data Available.
> 
> 
> Diffs
> -
> 
>   ambari-metrics/ambari-metrics-timelineservice/pom.xml 29f7812 
> 
> Diff: https://reviews.apache.org/r/48262/diff/
> 
> 
> Testing
> ---
> 
> FIX:
> 1. update Ambari Metrics collector pom.xml to use httpclient-4.2.5.jar.
> 2. without the fix, the collector lib contains httpclient-4.0.1.jar and 
> httpcore-4.2.5.jar.
> 3. with the fix, the collector lib will have httpclient-4.2.5.jar and 
> httpcore-4.2.4.jar.
> 
> TESTS:
> 1. The Ambari Metrics unit tests.
> 2. Also tested with HDP UI - after replacing httpclient-4.0.1.jar and 
> httpcore-4.2.5.jar with httpclient-4.2.5.jar and httpcore-4.2.4.jar, the 
> missing metrics show up.
> 
> 
> Thanks,
> 
> Qin Liu
> 
>

Re: Review Request 48262: AMBARI-16852: Update metrics collector pom.xml to use httpclient-4.2.5.jar

[Dmytro Sen]

> On Июнь 6, 2016, 4:32 п.п., Dmytro Sen wrote:
> > Ship It!

Committed to trunk and branch-2.4.
Please close this review board


- Dmytro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48262/#review136288
---


On Июнь 6, 2016, 6:26 д.п., Qin Liu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48262/
> ---
> 
> (Updated Июнь 6, 2016, 6:26 д.п.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Dmytro Sen, and Sid Wagle.
> 
> 
> Bugs: AMBARI-16852
> https://issues.apache.org/jira/browse/AMBARI-16852
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> httpclient-4.0.1.jar used by Ambari Metrics collector does not have method 
> URLEncodedUtils.parse (final URI uri, final String encoding). This will cause 
> the metrics not to be populated and the collector log to be flooded with the 
> following exception when the Ambari Metrics service is in distributed mode 
> and the user configures http authentication filter without enabling Kerberos:
> 
> 21:54:08,215 ERROR [1831490216@qtp-1883237069-7582] log:87 - Error for 
> /ws/v1/timeline/metrics
> java.lang.NoSuchMethodError: 
> org.apache.http.client.utils.URLEncodedUtils.parse(Ljava/lang/String;Ljava/nio/charset/Charset;)Ljava/util/List;
> at 
> org.apache.hadoop.security.authentication.server.PseudoAuthenticationHandler.getUserName(PseudoAuthenticationHandler.java:143)
> at 
> org.apache.hadoop.security.authentication.server.PseudoAuthenticationHandler.authenticate(PseudoAuthenticationHandler.java:179)
> at 
> org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:507)
> at 
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at 
> org.apache.hadoop.http.HttpServer2$QuotingInputFilter.doFilter(HttpServer2.java:1224)
> at 
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
> at 
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
> at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> 
> Steps to reproduce:
> 1. install a cluster with default from Ambari Web UI.
> 2. configure Ambari Metrics service in distributed mode.
> 3. Add the following properties in Custom core-site from Ambari Web UI:
> 1). 
> hadoop.http.authentication.signature.secret.file=/etc/security/http_secret.
> 2). 
> hadoop.http.filter.initializers=org.apache.hadoop.security.AuthenticationFilterInitialize.
> 4. create /etc/security/http_secret text file on each node.
> 5. restart HDFS, YARN, Mapreduce2, HBase, Kafka, and Ambari Metrics service.
> 6. Metrics on dashboard and HDFS, YARN, HBase, Kafka, and Amabari Metrics 
> will show No Data Available.
> 
> 
> Diffs
> -
> 
>   ambari-metrics/ambari-metrics-timelineservice/pom.xml 29f7812 
> 
> Diff: https://reviews.apache.org/r/48262/diff/
> 
> 
> Testing
> ---
> 
> FIX:
> 1. update Ambari Metrics collector pom.xml to use httpclient-4.2.5.jar.
> 2. without the fix, the collector lib contains httpclient-4.0.1.jar and 
> httpcore-4.2.5.jar.
> 3. with the fix, the collector lib will have httpclient-4.2.5.jar and 
> httpcore-4.2.4.jar.
> 
> TESTS:
> 1. The Ambari Metrics unit tests.
> 2. Also tested with HDP UI - after replacing httpclient-4.0.1.jar and 
> httpcore-4.2.5.jar with httpclient-4.2.5.jar and httpcore-4.2.4.jar, the 
> missing metrics show up.
> 
> 
> Thanks,
> 
> Qin Liu
> 
>

Re: Review Request 48207: Agents should automatically restart if there is a memory leak

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48207/#review136291
---


Ship it!




Ship It!

- Sumit Mohanty


On June 6, 2016, 4:29 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48207/
> ---
> 
> (Updated June 6, 2016, 4:29 p.m.)
> 
> 
> Review request for Ambari and Dmitro Lisnichenko.
> 
> 
> Bugs: AMBARI-17030
> https://issues.apache.org/jira/browse/AMBARI-17030
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add an agent capability to auto-restart when the memory threshold for the
> agent process exceeds a limit. This should be a configurable property (e.g.
> agent_memory_threshold) where any positive value, say in MB, is a limit at
> which agent should get restarted.
> 
> The restart itself should be executed when there are no
> EXECUTION/AUTO_EXECUTION commands in pipeline. So when the condition is
> reached for agent auto restart, a flag should be set to restart when agent is
> not doing any work.
> 
> A hard limit should also be set where the agent will be restarted even if
> there are commands in pipeline.
> 
> 
> Diffs
> -
> 
>   ambari-agent/conf/unix/ambari-agent.ini aacbb8a 
>   ambari-agent/src/main/python/ambari_agent/Controller.py 91bc586 
>   ambari-common/src/main/python/ambari_commons/os_utils.py 4a636d9 
> 
> Diff: https://reviews.apache.org/r/48207/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 48262: AMBARI-16852: Update metrics collector pom.xml to use httpclient-4.2.5.jar

[Aravindan Vijayan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48262/#review136289
---


Ship it!




Ship It!

- Aravindan Vijayan


On June 6, 2016, 6:26 a.m., Qin Liu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48262/
> ---
> 
> (Updated June 6, 2016, 6:26 a.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Dmytro Sen, and Sid Wagle.
> 
> 
> Bugs: AMBARI-16852
> https://issues.apache.org/jira/browse/AMBARI-16852
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> httpclient-4.0.1.jar used by Ambari Metrics collector does not have method 
> URLEncodedUtils.parse (final URI uri, final String encoding). This will cause 
> the metrics not to be populated and the collector log to be flooded with the 
> following exception when the Ambari Metrics service is in distributed mode 
> and the user configures http authentication filter without enabling Kerberos:
> 
> 21:54:08,215 ERROR [1831490216@qtp-1883237069-7582] log:87 - Error for 
> /ws/v1/timeline/metrics
> java.lang.NoSuchMethodError: 
> org.apache.http.client.utils.URLEncodedUtils.parse(Ljava/lang/String;Ljava/nio/charset/Charset;)Ljava/util/List;
> at 
> org.apache.hadoop.security.authentication.server.PseudoAuthenticationHandler.getUserName(PseudoAuthenticationHandler.java:143)
> at 
> org.apache.hadoop.security.authentication.server.PseudoAuthenticationHandler.authenticate(PseudoAuthenticationHandler.java:179)
> at 
> org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:507)
> at 
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at 
> org.apache.hadoop.http.HttpServer2$QuotingInputFilter.doFilter(HttpServer2.java:1224)
> at 
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
> at 
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
> at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> 
> Steps to reproduce:
> 1. install a cluster with default from Ambari Web UI.
> 2. configure Ambari Metrics service in distributed mode.
> 3. Add the following properties in Custom core-site from Ambari Web UI:
> 1). 
> hadoop.http.authentication.signature.secret.file=/etc/security/http_secret.
> 2). 
> hadoop.http.filter.initializers=org.apache.hadoop.security.AuthenticationFilterInitialize.
> 4. create /etc/security/http_secret text file on each node.
> 5. restart HDFS, YARN, Mapreduce2, HBase, Kafka, and Ambari Metrics service.
> 6. Metrics on dashboard and HDFS, YARN, HBase, Kafka, and Amabari Metrics 
> will show No Data Available.
> 
> 
> Diffs
> -
> 
>   ambari-metrics/ambari-metrics-timelineservice/pom.xml 29f7812 
> 
> Diff: https://reviews.apache.org/r/48262/diff/
> 
> 
> Testing
> ---
> 
> FIX:
> 1. update Ambari Metrics collector pom.xml to use httpclient-4.2.5.jar.
> 2. without the fix, the collector lib contains httpclient-4.0.1.jar and 
> httpcore-4.2.5.jar.
> 3. with the fix, the collector lib will have httpclient-4.2.5.jar and 
> httpcore-4.2.4.jar.
> 
> TESTS:
> 1. The Ambari Metrics unit tests.
> 2. Also tested with HDP UI - after replacing httpclient-4.0.1.jar and 
> httpcore-4.2.5.jar with httpclient-4.2.5.jar and httpcore-4.2.4.jar, the 
> missing metrics show up.
> 
> 
> Thanks,
> 
> Qin Liu
> 
>

Re: Review Request 48262: AMBARI-16852: Update metrics collector pom.xml to use httpclient-4.2.5.jar

[Dmytro Sen]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48262/#review136288
---


Ship it!




Ship It!

- Dmytro Sen


On Июнь 6, 2016, 6:26 д.п., Qin Liu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48262/
> ---
> 
> (Updated Июнь 6, 2016, 6:26 д.п.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Dmytro Sen, and Sid Wagle.
> 
> 
> Bugs: AMBARI-16852
> https://issues.apache.org/jira/browse/AMBARI-16852
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> httpclient-4.0.1.jar used by Ambari Metrics collector does not have method 
> URLEncodedUtils.parse (final URI uri, final String encoding). This will cause 
> the metrics not to be populated and the collector log to be flooded with the 
> following exception when the Ambari Metrics service is in distributed mode 
> and the user configures http authentication filter without enabling Kerberos:
> 
> 21:54:08,215 ERROR [1831490216@qtp-1883237069-7582] log:87 - Error for 
> /ws/v1/timeline/metrics
> java.lang.NoSuchMethodError: 
> org.apache.http.client.utils.URLEncodedUtils.parse(Ljava/lang/String;Ljava/nio/charset/Charset;)Ljava/util/List;
> at 
> org.apache.hadoop.security.authentication.server.PseudoAuthenticationHandler.getUserName(PseudoAuthenticationHandler.java:143)
> at 
> org.apache.hadoop.security.authentication.server.PseudoAuthenticationHandler.authenticate(PseudoAuthenticationHandler.java:179)
> at 
> org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:507)
> at 
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at 
> org.apache.hadoop.http.HttpServer2$QuotingInputFilter.doFilter(HttpServer2.java:1224)
> at 
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
> at 
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
> at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> 
> Steps to reproduce:
> 1. install a cluster with default from Ambari Web UI.
> 2. configure Ambari Metrics service in distributed mode.
> 3. Add the following properties in Custom core-site from Ambari Web UI:
> 1). 
> hadoop.http.authentication.signature.secret.file=/etc/security/http_secret.
> 2). 
> hadoop.http.filter.initializers=org.apache.hadoop.security.AuthenticationFilterInitialize.
> 4. create /etc/security/http_secret text file on each node.
> 5. restart HDFS, YARN, Mapreduce2, HBase, Kafka, and Ambari Metrics service.
> 6. Metrics on dashboard and HDFS, YARN, HBase, Kafka, and Amabari Metrics 
> will show No Data Available.
> 
> 
> Diffs
> -
> 
>   ambari-metrics/ambari-metrics-timelineservice/pom.xml 29f7812 
> 
> Diff: https://reviews.apache.org/r/48262/diff/
> 
> 
> Testing
> ---
> 
> FIX:
> 1. update Ambari Metrics collector pom.xml to use httpclient-4.2.5.jar.
> 2. without the fix, the collector lib contains httpclient-4.0.1.jar and 
> httpcore-4.2.5.jar.
> 3. with the fix, the collector lib will have httpclient-4.2.5.jar and 
> httpcore-4.2.4.jar.
> 
> TESTS:
> 1. The Ambari Metrics unit tests.
> 2. Also tested with HDP UI - after replacing httpclient-4.0.1.jar and 
> httpcore-4.2.5.jar with httpclient-4.2.5.jar and httpcore-4.2.4.jar, the 
> missing metrics show up.
> 
> 
> Thanks,
> 
> Qin Liu
> 
>

Re: Review Request 48207: Agents should automatically restart if there is a memory leak

[Andrew Onischuk]

> On June 6, 2016, 4:21 p.m., Sumit Mohanty wrote:
> > ambari-agent/conf/unix/ambari-agent.ini, line 37
> > 
> >
> > Whats the default behavior - looks like by default it should be 
> > disabled.
> > 
> > What is the memory consumption for an agent instance that is not 
> > leaking - assuming 20+ master/slave components on the host.

done


- Andrew


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48207/#review136283
---


On June 6, 2016, 4:29 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48207/
> ---
> 
> (Updated June 6, 2016, 4:29 p.m.)
> 
> 
> Review request for Ambari and Dmitro Lisnichenko.
> 
> 
> Bugs: AMBARI-17030
> https://issues.apache.org/jira/browse/AMBARI-17030
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add an agent capability to auto-restart when the memory threshold for the
> agent process exceeds a limit. This should be a configurable property (e.g.
> agent_memory_threshold) where any positive value, say in MB, is a limit at
> which agent should get restarted.
> 
> The restart itself should be executed when there are no
> EXECUTION/AUTO_EXECUTION commands in pipeline. So when the condition is
> reached for agent auto restart, a flag should be set to restart when agent is
> not doing any work.
> 
> A hard limit should also be set where the agent will be restarted even if
> there are commands in pipeline.
> 
> 
> Diffs
> -
> 
>   ambari-agent/conf/unix/ambari-agent.ini aacbb8a 
>   ambari-agent/src/main/python/ambari_agent/Controller.py 91bc586 
>   ambari-common/src/main/python/ambari_commons/os_utils.py 4a636d9 
> 
> Diff: https://reviews.apache.org/r/48207/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 48207: Agents should automatically restart if there is a memory leak

[Andrew Onischuk]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48207/
---

(Updated June 6, 2016, 4:29 p.m.)


Review request for Ambari and Dmitro Lisnichenko.


Bugs: AMBARI-17030
https://issues.apache.org/jira/browse/AMBARI-17030


Repository: ambari


Description
---

Add an agent capability to auto-restart when the memory threshold for the
agent process exceeds a limit. This should be a configurable property (e.g.
agent_memory_threshold) where any positive value, say in MB, is a limit at
which agent should get restarted.

The restart itself should be executed when there are no
EXECUTION/AUTO_EXECUTION commands in pipeline. So when the condition is
reached for agent auto restart, a flag should be set to restart when agent is
not doing any work.

A hard limit should also be set where the agent will be restarted even if
there are commands in pipeline.


Diffs (updated)
-

  ambari-agent/conf/unix/ambari-agent.ini aacbb8a 
  ambari-agent/src/main/python/ambari_agent/Controller.py 91bc586 
  ambari-common/src/main/python/ambari_commons/os_utils.py 4a636d9 

Diff: https://reviews.apache.org/r/48207/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Re: Review Request 48282: YARN service check fails if there is no queue named "default"

[Dmytro Sen]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48282/#review136282
---


Ship it!




Ship It!

- Dmytro Sen


On Июнь 6, 2016, 3:18 п.п., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48282/
> ---
> 
> (Updated Июнь 6, 2016, 3:18 п.п.)
> 
> 
> Review request for Ambari and Dmytro Sen.
> 
> 
> Bugs: AMBARI-15557
> https://issues.apache.org/jira/browse/AMBARI-15557
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Ambari runs this command when running YARN service check. 
> 
> yarn org.apache.hadoop.yarn.applications.distributedshell.Client 
> -shell_command ls -num_containers 1 -jar 
> /usr/hdp/current/hadoop-yarn-client/hadoop-yarn-applications-distributedshell.j
>  
> 
> In the logs , we can see that its looking for queue named "default" 
> 
> 15/12/23 19:09:49 INFO distributedshell.Client: Queue info, 
> queueName=default, 
> 
> I was able to reproduce this by changing 
> 
> this 
> 
> yarn.scheduler.capacity.default.minimum-user-limit-percent=100 
> yarn.scheduler.capacity.maximum-am-resource-percent=0.2 
> yarn.scheduler.capacity.maximum-applications=1 
> yarn.scheduler.capacity.node-locality-delay=40 
> yarn.scheduler.capacity.root.accessible-node-labels=* 
> yarn.scheduler.capacity.root.acl_administer_queue=* 
> yarn.scheduler.capacity.root.capacity=100 
> yarn.scheduler.capacity.root.default.acl_administer_jobs=* 
> yarn.scheduler.capacity.root.default.acl_submit_applications=* 
> yarn.scheduler.capacity.root.default.capacity=100 
> yarn.scheduler.capacity.root.default.maximum-capacity=100 
> yarn.scheduler.capacity.root.default.state=RUNNING 
> yarn.scheduler.capacity.root.default.user-limit-factor=1 
> yarn.scheduler.capacity.root.queues=default 
> 
> to 
> 
> yarn.scheduler.capacity.default12345.minimum-user-limit-percent=100 
> yarn.scheduler.capacity.maximum-am-resource-percent=0.2 
> yarn.scheduler.capacity.maximum-applications=1 
> yarn.scheduler.capacity.node-locality-delay=40 
> yarn.scheduler.capacity.root.accessible-node-labels=* 
> yarn.scheduler.capacity.root.acl_administer_queue=* 
> yarn.scheduler.capacity.root.capacity=100 
> yarn.scheduler.capacity.root.default12345.acl_administer_jobs=* 
> yarn.scheduler.capacity.root.default12345.acl_submit_applications=* 
> yarn.scheduler.capacity.root.default12345.capacity=100 
> yarn.scheduler.capacity.root.default12345.maximum-capacity=100 
> yarn.scheduler.capacity.root.default12345.state=RUNNING 
> yarn.scheduler.capacity.root.default12345.user-limit-factor=1 
> yarn.scheduler.capacity.root.queues=default12345
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/configuration/yarn-env.xml
>  648c32e 
>   
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py
>  a63abd5 
>   
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/service_check.py
>  26f764e 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
> 1dedf3d 
>   ambari-server/src/main/resources/stacks/HDP/2.2/services/stack_advisor.py 
> 03e8150 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
> 379c268 
>   ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py 
> 3744c26 
>   ambari-server/src/test/python/stacks/2.5/common/test_stack_advisor.py 
> 46e9f47 
> 
> Diff: https://reviews.apache.org/r/48282/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 48207: Agents should automatically restart if there is a memory leak

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48207/#review136283
---




ambari-agent/conf/unix/ambari-agent.ini (line 37)


Whats the default behavior - looks like by default it should be disabled.

What is the memory consumption for an agent instance that is not leaking - 
assuming 20+ master/slave components on the host.


- Sumit Mohanty


On June 6, 2016, 4:13 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48207/
> ---
> 
> (Updated June 6, 2016, 4:13 p.m.)
> 
> 
> Review request for Ambari and Dmitro Lisnichenko.
> 
> 
> Bugs: AMBARI-17030
> https://issues.apache.org/jira/browse/AMBARI-17030
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add an agent capability to auto-restart when the memory threshold for the
> agent process exceeds a limit. This should be a configurable property (e.g.
> agent_memory_threshold) where any positive value, say in MB, is a limit at
> which agent should get restarted.
> 
> The restart itself should be executed when there are no
> EXECUTION/AUTO_EXECUTION commands in pipeline. So when the condition is
> reached for agent auto restart, a flag should be set to restart when agent is
> not doing any work.
> 
> A hard limit should also be set where the agent will be restarted even if
> there are commands in pipeline.
> 
> 
> Diffs
> -
> 
>   ambari-agent/conf/unix/ambari-agent.ini aacbb8a 
>   ambari-agent/src/main/python/ambari_agent/Controller.py 91bc586 
>   ambari-common/src/main/python/ambari_commons/os_utils.py 4a636d9 
> 
> Diff: https://reviews.apache.org/r/48207/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 48207: Agents should automatically restart if there is a memory leak

[Andrew Onischuk]

> On June 3, 2016, 8:15 p.m., Alejandro Fernandez wrote:
> > ambari-agent/src/main/python/ambari_agent/Controller.py, line 290
> > 
> >
> > Restarting is dangerous to do, especially if it happens during 
> > kerberization or RU/EU. If there's a memory leak, then that's the problem 
> > we should be fixing.
> 
> Sumit Mohanty wrote:
> Agree. Andrew can you open an JIRA and add details of investigations 
> going on for memory leak

Opened an apache jira https://issues.apache.org/jira/browse/AMBARI-17065 to 
track the actual issue


The concern about RU/EU/kerberos should not be a problem because in this patch 
we do restart only if there are no commands queued on the agent.


- Andrew


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48207/#review136102
---


On June 6, 2016, 4:13 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48207/
> ---
> 
> (Updated June 6, 2016, 4:13 p.m.)
> 
> 
> Review request for Ambari and Dmitro Lisnichenko.
> 
> 
> Bugs: AMBARI-17030
> https://issues.apache.org/jira/browse/AMBARI-17030
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add an agent capability to auto-restart when the memory threshold for the
> agent process exceeds a limit. This should be a configurable property (e.g.
> agent_memory_threshold) where any positive value, say in MB, is a limit at
> which agent should get restarted.
> 
> The restart itself should be executed when there are no
> EXECUTION/AUTO_EXECUTION commands in pipeline. So when the condition is
> reached for agent auto restart, a flag should be set to restart when agent is
> not doing any work.
> 
> A hard limit should also be set where the agent will be restarted even if
> there are commands in pipeline.
> 
> 
> Diffs
> -
> 
>   ambari-agent/conf/unix/ambari-agent.ini aacbb8a 
>   ambari-agent/src/main/python/ambari_agent/Controller.py 91bc586 
>   ambari-common/src/main/python/ambari_commons/os_utils.py 4a636d9 
> 
> Diff: https://reviews.apache.org/r/48207/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 48207: Agents should automatically restart if there is a memory leak

[Andrew Onischuk]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48207/
---

(Updated June 6, 2016, 4:13 p.m.)


Review request for Ambari and Dmitro Lisnichenko.


Bugs: AMBARI-17030
https://issues.apache.org/jira/browse/AMBARI-17030


Repository: ambari


Description
---

Add an agent capability to auto-restart when the memory threshold for the
agent process exceeds a limit. This should be a configurable property (e.g.
agent_memory_threshold) where any positive value, say in MB, is a limit at
which agent should get restarted.

The restart itself should be executed when there are no
EXECUTION/AUTO_EXECUTION commands in pipeline. So when the condition is
reached for agent auto restart, a flag should be set to restart when agent is
not doing any work.

A hard limit should also be set where the agent will be restarted even if
there are commands in pipeline.


Diffs (updated)
-

  ambari-agent/conf/unix/ambari-agent.ini aacbb8a 
  ambari-agent/src/main/python/ambari_agent/Controller.py 91bc586 
  ambari-common/src/main/python/ambari_commons/os_utils.py 4a636d9 

Diff: https://reviews.apache.org/r/48207/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Re: Review Request 48207: Agents should automatically restart if there is a memory leak

[Andrew Onischuk]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48207/
---

(Updated June 6, 2016, 4:04 p.m.)


Review request for Ambari and Dmitro Lisnichenko.


Bugs: AMBARI-17030
https://issues.apache.org/jira/browse/AMBARI-17030


Repository: ambari


Description
---

Add an agent capability to auto-restart when the memory threshold for the
agent process exceeds a limit. This should be a configurable property (e.g.
agent_memory_threshold) where any positive value, say in MB, is a limit at
which agent should get restarted.

The restart itself should be executed when there are no
EXECUTION/AUTO_EXECUTION commands in pipeline. So when the condition is
reached for agent auto restart, a flag should be set to restart when agent is
not doing any work.

A hard limit should also be set where the agent will be restarted even if
there are commands in pipeline.


Diffs (updated)
-

  ambari-agent/conf/unix/ambari-agent.ini aacbb8a 
  ambari-agent/src/main/python/ambari_agent/Controller.py 91bc586 
  ambari-common/src/main/python/ambari_commons/os_utils.py 4a636d9 

Diff: https://reviews.apache.org/r/48207/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Re: Review Request 48283: Namenode failed to start while installing a cluster from UI

[Dmytro Sen]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48283/#review136274
---


Ship it!




Ship It!

- Dmytro Sen


On Июнь 6, 2016, 3:31 п.п., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48283/
> ---
> 
> (Updated Июнь 6, 2016, 3:31 п.п.)
> 
> 
> Review request for Ambari and Dmytro Sen.
> 
> 
> Bugs: AMBARI-17062
> https://issues.apache.org/jira/browse/AMBARI-17062
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> STR: Install a cluster from UI with HDFS, ZK and YARN  
> Namenode fails to start  
> The cluster is live here: 
> 
> 
> 
> 
> Traceback (most recent call last):
>   File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py",
>  line 414, in 
> NameNode().execute()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 257, in execute
> method(env)
>   File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py",
>  line 101, in start
> upgrade_suspended=params.upgrade_suspended, env=env)
>   File 
> "/usr/lib/python2.6/site-packages/ambari_commons/os_family_impl.py", line 89, 
> in thunk
> return fn(*args, **kwargs)
>   File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py",
>  line 155, in namenode
> create_log_dir=True
>   File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/utils.py",
>  line 269, in service
> Execute(daemon_cmd, not_if=process_id_exists_command, 
> environment=hadoop_env_exports)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 
> 155, in __init__
> self.env.run()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
> line 160, in run
> self.run_action(resource, action)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
> line 124, in run_action
> provider_action()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py",
>  line 273, in action_run
> tries=self.resource.tries, try_sleep=self.resource.try_sleep)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
> 70, in inner
> result = function(command, **kwargs)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
> 92, in checked_call
> tries=tries, try_sleep=try_sleep)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
> 140, in _call_wrapper
> result = _call(command, **kwargs_copy)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
> 293, in _call
> raise Fail(err_msg)
> resource_management.core.exceptions.Fail: Execution of 'ambari-sudo.sh su 
> hdfs -l -s /bin/bash -c 'ulimit -c unlimited ;  
> /usr/hdp/current/hadoop-client/sbin/hadoop-daemon.sh --config 
> /usr/hdp/current/hadoop-client/conf start namenode'' returned 1.  
> Hortonworks #
> This is MOTD message, added for testing in qe infra
> starting namenode, logging to 
> /grid/0/log/hdfs/hdfs/hadoop-hdfs-namenode-os-r7-vsrfou-ambari-serv-11r-1.out
> log4j:ERROR setFile(null,true) call failed.
> java.io.FileNotFoundException: ./nm-audit.log (Permission denied)
>   at java.io.FileOutputStream.open(Native Method)
>   at java.io.FileOutputStream.(FileOutputStream.java:221)
>   at java.io.FileOutputStream.(FileOutputStream.java:142)
>   at org.apache.log4j.FileAppender.setFile(FileAppender.java:294)
>   at org.apache.log4j.FileAppender.activateOptions(FileAppender.java:165)
>   at 
> org.apache.log4j.DailyRollingFileAppender.activateOptions(DailyRollingFileAppender.java:223)
>   at 
> org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:307)
>   at 
> org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:172)
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/python/ambari_server/serverSetup.py d862411 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/configuration/hadoop-env.xml
>  fbcd0ec 
>   
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/configuration/yarn-env.xml
>  648c32e 
>   
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/configuration/yarn-log4j.xml
>  7fded04 
>   
> 

Review Request 48284: Retrieve specific metrics when Ambari queries NameNode HA states

[Andrew Onischuk]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48284/
---

Review request for Ambari and Vitalyi Brodetskyi.


Bugs: AMBARI-17063
https://issues.apache.org/jira/browse/AMBARI-17063


Repository: ambari


Description
---

This is a follow-on jira of BUG-53459: when Ambari queries NameNode's HA
states, currently it retrieves a group of metrics including some unnecessary
ones that may compete for NameNode lock. It would be better for Ambari to only
retrieve HA state metrics from NameNode.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/jmx/JMXPropertyProvider.java
 e1a5e22 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/jmx/TestStreamProvider.java
 8819991 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/metrics/JMXPropertyProviderTest.java
 4adea20 
  ambari-server/src/test/resources/hdfs_namenode_jmx_ha_only.json PRE-CREATION 

Diff: https://reviews.apache.org/r/48284/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Review Request 48283: Namenode failed to start while installing a cluster from UI

[Andrew Onischuk]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48283/
---

Review request for Ambari and Dmytro Sen.


Bugs: AMBARI-17062
https://issues.apache.org/jira/browse/AMBARI-17062


Repository: ambari


Description
---

STR: Install a cluster from UI with HDFS, ZK and YARN  
Namenode fails to start  
The cluster is live here: 




Traceback (most recent call last):
  File 
"/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py",
 line 414, in 
NameNode().execute()
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
 line 257, in execute
method(env)
  File 
"/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py",
 line 101, in start
upgrade_suspended=params.upgrade_suspended, env=env)
  File "/usr/lib/python2.6/site-packages/ambari_commons/os_family_impl.py", 
line 89, in thunk
return fn(*args, **kwargs)
  File 
"/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py",
 line 155, in namenode
create_log_dir=True
  File 
"/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/utils.py",
 line 269, in service
Execute(daemon_cmd, not_if=process_id_exists_command, 
environment=hadoop_env_exports)
  File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", 
line 155, in __init__
self.env.run()
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
line 160, in run
self.run_action(resource, action)
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
line 124, in run_action
provider_action()
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py",
 line 273, in action_run
tries=self.resource.tries, try_sleep=self.resource.try_sleep)
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 70, 
in inner
result = function(command, **kwargs)
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 92, 
in checked_call
tries=tries, try_sleep=try_sleep)
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 140, 
in _call_wrapper
result = _call(command, **kwargs_copy)
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 293, 
in _call
raise Fail(err_msg)
resource_management.core.exceptions.Fail: Execution of 'ambari-sudo.sh su 
hdfs -l -s /bin/bash -c 'ulimit -c unlimited ;  
/usr/hdp/current/hadoop-client/sbin/hadoop-daemon.sh --config 
/usr/hdp/current/hadoop-client/conf start namenode'' returned 1.  
Hortonworks #
This is MOTD message, added for testing in qe infra
starting namenode, logging to 
/grid/0/log/hdfs/hdfs/hadoop-hdfs-namenode-os-r7-vsrfou-ambari-serv-11r-1.out
log4j:ERROR setFile(null,true) call failed.
java.io.FileNotFoundException: ./nm-audit.log (Permission denied)
at java.io.FileOutputStream.open(Native Method)
at java.io.FileOutputStream.(FileOutputStream.java:221)
at java.io.FileOutputStream.(FileOutputStream.java:142)
at org.apache.log4j.FileAppender.setFile(FileAppender.java:294)
at org.apache.log4j.FileAppender.activateOptions(FileAppender.java:165)
at 
org.apache.log4j.DailyRollingFileAppender.activateOptions(DailyRollingFileAppender.java:223)
at 
org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:307)
at 
org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:172)


Diffs
-

  ambari-server/src/main/python/ambari_server/serverSetup.py d862411 
  
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/configuration/hadoop-env.xml
 fbcd0ec 
  
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/configuration/yarn-env.xml
 648c32e 
  
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/configuration/yarn-log4j.xml
 7fded04 
  
ambari-server/src/main/resources/stacks/HDP/2.1/services/YARN/configuration/yarn-env.xml
 8cf516c 
  
ambari-server/src/main/resources/stacks/HDP/2.3/services/YARN/configuration/yarn-env.xml
 b517f91 
  
ambari-server/src/main/resources/stacks/HDP/2.3/services/YARN/configuration/yarn-log4j.xml
 ef84187 

Diff: https://reviews.apache.org/r/48283/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Review Request 48273: AMBARI-17054 : Configure Atlas Ranger Plugin

[Gautam Borad]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48273/
---

Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Srimanth 
Gunturi, and Velmurugan Periasamy.


Bugs: AMBARI-17054
https://issues.apache.org/jira/browse/AMBARI-17054


Repository: ambari


Description
---

Patch contains changes to support Ranger Atlas Plugin from Ambari.


Diffs
-

  
ambari-common/src/main/python/resource_management/libraries/functions/constants.py
 34d1a1a 
  
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata_server.py
 1e9e7a7 
  
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py
 a79a456 
  
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/setup_ranger_atlas.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/status_params.py
 4c54214 
  
ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-env.xml
 4db7f45 
  
ambari-server/src/main/resources/common-services/RANGER/0.6.0/themes/theme_version_3.json
 0f7b0c0 
  
ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_features.json
 8b669e8 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/configuration/application-properties.xml
 98cbc30 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/configuration/ranger-atlas-audit.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/configuration/ranger-atlas-plugin-properties.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/configuration/ranger-atlas-policymgr-ssl.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/configuration/ranger-atlas-security.xml
 PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
379c268 

Diff: https://reviews.apache.org/r/48273/diff/


Testing
---

Verified : 

1) Enable / disable actions for Ranger Atlas Plugin
2) Checked Ranger plugin comminication. 
3) Checked policy enforecements on Atlas based on Ranger's policies.
4) Verified stack advisor recommendations to chnange following properties : 

atlas.authorizer.impl=org.apache.ranger.authorization.atlas.authorizer.RangerAtlasAuthorizer
ranger-atlas-plugin-enabled=yes
5) Verified Ranger service creation (while enabling Ranger Atlas Plugin)


Thanks,

Gautam Borad

Review Request 48282: YARN service check fails if there is no queue named "default"

[Andrew Onischuk]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48282/
---

Review request for Ambari and Dmytro Sen.


Bugs: AMBARI-15557
https://issues.apache.org/jira/browse/AMBARI-15557


Repository: ambari


Description
---

Ambari runs this command when running YARN service check. 

yarn org.apache.hadoop.yarn.applications.distributedshell.Client -shell_command 
ls -num_containers 1 -jar 
/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-applications-distributedshell.j 

In the logs , we can see that its looking for queue named "default" 

15/12/23 19:09:49 INFO distributedshell.Client: Queue info, queueName=default, 

I was able to reproduce this by changing 

this 

yarn.scheduler.capacity.default.minimum-user-limit-percent=100 
yarn.scheduler.capacity.maximum-am-resource-percent=0.2 
yarn.scheduler.capacity.maximum-applications=1 
yarn.scheduler.capacity.node-locality-delay=40 
yarn.scheduler.capacity.root.accessible-node-labels=* 
yarn.scheduler.capacity.root.acl_administer_queue=* 
yarn.scheduler.capacity.root.capacity=100 
yarn.scheduler.capacity.root.default.acl_administer_jobs=* 
yarn.scheduler.capacity.root.default.acl_submit_applications=* 
yarn.scheduler.capacity.root.default.capacity=100 
yarn.scheduler.capacity.root.default.maximum-capacity=100 
yarn.scheduler.capacity.root.default.state=RUNNING 
yarn.scheduler.capacity.root.default.user-limit-factor=1 
yarn.scheduler.capacity.root.queues=default 

to 

yarn.scheduler.capacity.default12345.minimum-user-limit-percent=100 
yarn.scheduler.capacity.maximum-am-resource-percent=0.2 
yarn.scheduler.capacity.maximum-applications=1 
yarn.scheduler.capacity.node-locality-delay=40 
yarn.scheduler.capacity.root.accessible-node-labels=* 
yarn.scheduler.capacity.root.acl_administer_queue=* 
yarn.scheduler.capacity.root.capacity=100 
yarn.scheduler.capacity.root.default12345.acl_administer_jobs=* 
yarn.scheduler.capacity.root.default12345.acl_submit_applications=* 
yarn.scheduler.capacity.root.default12345.capacity=100 
yarn.scheduler.capacity.root.default12345.maximum-capacity=100 
yarn.scheduler.capacity.root.default12345.state=RUNNING 
yarn.scheduler.capacity.root.default12345.user-limit-factor=1 
yarn.scheduler.capacity.root.queues=default12345


Diffs
-

  
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/configuration/yarn-env.xml
 648c32e 
  
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py
 a63abd5 
  
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/service_check.py
 26f764e 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
1dedf3d 
  ambari-server/src/main/resources/stacks/HDP/2.2/services/stack_advisor.py 
03e8150 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
379c268 
  ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py 
3744c26 
  ambari-server/src/test/python/stacks/2.5/common/test_stack_advisor.py 46e9f47 

Diff: https://reviews.apache.org/r/48282/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Re: Review Request 48044: Provide context for hdp-select failures during ambari component install

[Laszlo Puskas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48044/
---

(Updated June 6, 2016, 3:15 p.m.)


Review request for Ambari, Andrew Onischuk, Oliver Szabo, Sumit Mohanty, and 
Sebastian Toader.


Bugs: AMBARI-16952
https://issues.apache.org/jira/browse/AMBARI-16952


Repository: ambari


Description
---

In case the hdp-select command fails during a service / component installation 
there's no contextual information about the cause of the failure.
This issue is for logging information about the machine on which the hdp-select 
command fails.
This solution wraps hdp-select command calls in a try/catch block and logs 
failure / hdp installationrelated information.


Diffs
-

  
ambari-common/src/main/python/resource_management/libraries/functions/hdp_select.py
 9a3201e 

Diff: https://reviews.apache.org/r/48044/diff/


Testing (updated)
---

Unit tests passed.
Manual testing underway please don't commit yet.
// found some major differences between the branches / investigating it


Thanks,

Laszlo Puskas

Re: Review Request 48274: Show better error message when VIEW USER can not access CAPACITY SCHEDULER View

[Nitiraj Rathore]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48274/#review136270
---


Ship it!




Ship It!

- Nitiraj Rathore


On June 6, 2016, 12:23 p.m., Gaurav Nagar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48274/
> ---
> 
> (Updated June 6, 2016, 12:23 p.m.)
> 
> 
> Review request for Ambari, DIPAYAN BHOWMICK, Nitiraj Rathore, Pallav 
> Kulshreshtha, Rohit Choudhary, and Ashwin Rajeev.
> 
> 
> Bugs: AMBARI-17055
> https://issues.apache.org/jira/browse/AMBARI-17055
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Added error message when user does not have read permission on configurations.
> Added message when view is in read only mode.
> 
> 
> Diffs
> -
> 
>   
> contrib/views/capacity-scheduler/src/main/java/org/apache/ambari/view/capacityscheduler/ConfigurationService.java
>  42076fe 
>   
> contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/queues.hbs
>  94cb421 
>   
> contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/queues/index.hbs
>  1c468b5 
>   
> contrib/views/utils/src/main/java/org/apache/ambari/view/utils/ambari/AmbariApi.java
>  86d8434 
> 
> Diff: https://reviews.apache.org/r/48274/diff/
> 
> 
> Testing
> ---
> 
> Manully Tested
> 
> 
> Thanks,
> 
> Gaurav Nagar
> 
>

Re: Review Request 48274: Show better error message when VIEW USER can not access CAPACITY SCHEDULER View

[Nitiraj Rathore]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48274/#review136269
---


Ship it!




Ship It!

- Nitiraj Rathore


On June 6, 2016, 12:23 p.m., Gaurav Nagar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48274/
> ---
> 
> (Updated June 6, 2016, 12:23 p.m.)
> 
> 
> Review request for Ambari, DIPAYAN BHOWMICK, Nitiraj Rathore, Pallav 
> Kulshreshtha, Rohit Choudhary, and Ashwin Rajeev.
> 
> 
> Bugs: AMBARI-17055
> https://issues.apache.org/jira/browse/AMBARI-17055
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Added error message when user does not have read permission on configurations.
> Added message when view is in read only mode.
> 
> 
> Diffs
> -
> 
>   
> contrib/views/capacity-scheduler/src/main/java/org/apache/ambari/view/capacityscheduler/ConfigurationService.java
>  42076fe 
>   
> contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/queues.hbs
>  94cb421 
>   
> contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/queues/index.hbs
>  1c468b5 
>   
> contrib/views/utils/src/main/java/org/apache/ambari/view/utils/ambari/AmbariApi.java
>  86d8434 
> 
> Diff: https://reviews.apache.org/r/48274/diff/
> 
> 
> Testing
> ---
> 
> Manully Tested
> 
> 
> Thanks,
> 
> Gaurav Nagar
> 
>

Re: Review Request 48044: Provide context for hdp-select failures during ambari component install

[Laszlo Puskas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48044/
---

(Updated June 6, 2016, 3:01 p.m.)


Review request for Ambari, Andrew Onischuk, Oliver Szabo, Sumit Mohanty, and 
Sebastian Toader.


Bugs: AMBARI-16952
https://issues.apache.org/jira/browse/AMBARI-16952


Repository: ambari


Description
---

In case the hdp-select command fails during a service / component installation 
there's no contextual information about the cause of the failure.
This issue is for logging information about the machine on which the hdp-select 
command fails.
This solution wraps hdp-select command calls in a try/catch block and logs 
failure / hdp installationrelated information.


Diffs
-

  
ambari-common/src/main/python/resource_management/libraries/functions/hdp_select.py
 9a3201e 

Diff: https://reviews.apache.org/r/48044/diff/


Testing (updated)
---

Unit tests passed.
Manual testing underway please don't commit yet.


Thanks,

Laszlo Puskas

Re: Review Request 48044: Provide context for hdp-select failures during ambari component install

[Laszlo Puskas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48044/
---

(Updated June 6, 2016, 3 p.m.)


Review request for Ambari, Andrew Onischuk, Oliver Szabo, Sumit Mohanty, and 
Sebastian Toader.


Bugs: AMBARI-16952
https://issues.apache.org/jira/browse/AMBARI-16952


Repository: ambari


Description
---

In case the hdp-select command fails during a service / component installation 
there's no contextual information about the cause of the failure.
This issue is for logging information about the machine on which the hdp-select 
command fails.
This solution wraps hdp-select command calls in a try/catch block and logs 
failure / hdp installationrelated information.


Diffs
-

  
ambari-common/src/main/python/resource_management/libraries/functions/hdp_select.py
 9a3201e 

Diff: https://reviews.apache.org/r/48044/diff/


Testing (updated)
---

Unit tests passed.
Manual testing underway.


Thanks,

Laszlo Puskas

Re: Review Request 48281: [ambari] ambari-agent restart is failing due to unsupported locale setting

[Vitalyi Brodetskyi]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48281/#review136264
---


Ship it!




Ship It!

- Vitalyi Brodetskyi


On Червень 6, 2016, 2:47 після полудня, Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48281/
> ---
> 
> (Updated Червень 6, 2016, 2:47 після полудня)
> 
> 
> Review request for Ambari and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-17061
> https://issues.apache.org/jira/browse/AMBARI-17061
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> this issue I faced some 5-6 days ago , I was not able to reproduce the same.  
> Again i am facing the same issue, incident is as follows:  
> 1) ambari-agent heartbeat lost  
> 2) checked the ambari-agent logs couldn't find any valid reason  
> 3) so restarted the ambari-agent
> 
> 
> 
> 
> [root@ds-xa-test-1-re-1 bin]# ambari-agent start
> Verifying Python version compatibility...
> Using python  /usr/bin/python
> Checking for previously running Ambari Agent...
> Starting ambari-agent
> Verifying ambari-agent process status...
> Ambari Agent successfully started
> Agent PID at: /var/run/ambari-agent/ambari-agent.pid
> Agent out at: /var/log/ambari-agent/ambari-agent.out
> Agent log at: /var/log/ambari-agent/ambari-agent.log
> 
> 
> **it says ambari-agent successfull started but it does not start actually and 
> throws unsupported locale type exception in agent.out**
> 
> following is the exception in ambari-agent.out
> 
> 
> 
> 
>   File "/usr/lib/python2.6/site-packages/ambari_agent/main.py", line 54, 
> in 
> locale.setlocale(locale.LC_ALL, '')
>   File "/usr/lib64/python2.6/locale.py", line 513, in setlocale
> return _setlocale(category, locale)
> locale.Error: unsupported locale setting
> ^[[?1034h
> 
> 
> I checked the locale
> 
> 
> 
> 
> [root@ds-xa-test-1-re-1 bin]# locale
> locale: Cannot set LC_CTYPE to default locale: No such file or directory
> locale: Cannot set LC_ALL to default locale: No such file or directory
> LANG=en_US.UTF-8
> LC_CTYPE=UTF-8
> LC_NUMERIC="en_US.UTF-8"
> LC_TIME="en_US.UTF-8"
> LC_COLLATE="en_US.UTF-8"
> LC_MONETARY="en_US.UTF-8"
> LC_MESSAGES="en_US.UTF-8"
> LC_PAPER="en_US.UTF-8"
> LC_NAME="en_US.UTF-8"
> LC_ADDRESS="en_US.UTF-8"
> LC_TELEPHONE="en_US.UTF-8"
> LC_MEASUREMENT="en_US.UTF-8"
> LC_IDENTIFICATION="en_US.UTF-8"
> LC_ALL=
> 
> 
> since LC_ALL was set to empty so set the export LC_ALL=C
> 
> then restarted the ranger and it was successful.
> 
> again facing the same issue:
> 
> 
> build detail:  
> ambari-agent-2.4.0.0-625.x86_64
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/main.py 32e522d 
> 
> Diff: https://reviews.apache.org/r/48281/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 48281: [ambari] ambari-agent restart is failing due to unsupported locale setting

[Vitalyi Brodetskyi]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48281/#review136263
---


Ship it!




Ship It!

- Vitalyi Brodetskyi


On Червень 6, 2016, 2:47 після полудня, Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48281/
> ---
> 
> (Updated Червень 6, 2016, 2:47 після полудня)
> 
> 
> Review request for Ambari and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-17061
> https://issues.apache.org/jira/browse/AMBARI-17061
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> this issue I faced some 5-6 days ago , I was not able to reproduce the same.  
> Again i am facing the same issue, incident is as follows:  
> 1) ambari-agent heartbeat lost  
> 2) checked the ambari-agent logs couldn't find any valid reason  
> 3) so restarted the ambari-agent
> 
> 
> 
> 
> [root@ds-xa-test-1-re-1 bin]# ambari-agent start
> Verifying Python version compatibility...
> Using python  /usr/bin/python
> Checking for previously running Ambari Agent...
> Starting ambari-agent
> Verifying ambari-agent process status...
> Ambari Agent successfully started
> Agent PID at: /var/run/ambari-agent/ambari-agent.pid
> Agent out at: /var/log/ambari-agent/ambari-agent.out
> Agent log at: /var/log/ambari-agent/ambari-agent.log
> 
> 
> **it says ambari-agent successfull started but it does not start actually and 
> throws unsupported locale type exception in agent.out**
> 
> following is the exception in ambari-agent.out
> 
> 
> 
> 
>   File "/usr/lib/python2.6/site-packages/ambari_agent/main.py", line 54, 
> in 
> locale.setlocale(locale.LC_ALL, '')
>   File "/usr/lib64/python2.6/locale.py", line 513, in setlocale
> return _setlocale(category, locale)
> locale.Error: unsupported locale setting
> ^[[?1034h
> 
> 
> I checked the locale
> 
> 
> 
> 
> [root@ds-xa-test-1-re-1 bin]# locale
> locale: Cannot set LC_CTYPE to default locale: No such file or directory
> locale: Cannot set LC_ALL to default locale: No such file or directory
> LANG=en_US.UTF-8
> LC_CTYPE=UTF-8
> LC_NUMERIC="en_US.UTF-8"
> LC_TIME="en_US.UTF-8"
> LC_COLLATE="en_US.UTF-8"
> LC_MONETARY="en_US.UTF-8"
> LC_MESSAGES="en_US.UTF-8"
> LC_PAPER="en_US.UTF-8"
> LC_NAME="en_US.UTF-8"
> LC_ADDRESS="en_US.UTF-8"
> LC_TELEPHONE="en_US.UTF-8"
> LC_MEASUREMENT="en_US.UTF-8"
> LC_IDENTIFICATION="en_US.UTF-8"
> LC_ALL=
> 
> 
> since LC_ALL was set to empty so set the export LC_ALL=C
> 
> then restarted the ranger and it was successful.
> 
> again facing the same issue:
> 
> 
> build detail:  
> ambari-agent-2.4.0.0-625.x86_64
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/main.py 32e522d 
> 
> Diff: https://reviews.apache.org/r/48281/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Review Request 48281: [ambari] ambari-agent restart is failing due to unsupported locale setting

[Andrew Onischuk]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48281/
---

Review request for Ambari and Vitalyi Brodetskyi.


Bugs: AMBARI-17061
https://issues.apache.org/jira/browse/AMBARI-17061


Repository: ambari


Description
---

this issue I faced some 5-6 days ago , I was not able to reproduce the same.  
Again i am facing the same issue, incident is as follows:  
1) ambari-agent heartbeat lost  
2) checked the ambari-agent logs couldn't find any valid reason  
3) so restarted the ambari-agent




[root@ds-xa-test-1-re-1 bin]# ambari-agent start
Verifying Python version compatibility...
Using python  /usr/bin/python
Checking for previously running Ambari Agent...
Starting ambari-agent
Verifying ambari-agent process status...
Ambari Agent successfully started
Agent PID at: /var/run/ambari-agent/ambari-agent.pid
Agent out at: /var/log/ambari-agent/ambari-agent.out
Agent log at: /var/log/ambari-agent/ambari-agent.log


**it says ambari-agent successfull started but it does not start actually and 
throws unsupported locale type exception in agent.out**

following is the exception in ambari-agent.out




  File "/usr/lib/python2.6/site-packages/ambari_agent/main.py", line 54, in 

locale.setlocale(locale.LC_ALL, '')
  File "/usr/lib64/python2.6/locale.py", line 513, in setlocale
return _setlocale(category, locale)
locale.Error: unsupported locale setting
^[[?1034h


I checked the locale




[root@ds-xa-test-1-re-1 bin]# locale
locale: Cannot set LC_CTYPE to default locale: No such file or directory
locale: Cannot set LC_ALL to default locale: No such file or directory
LANG=en_US.UTF-8
LC_CTYPE=UTF-8
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=


since LC_ALL was set to empty so set the export LC_ALL=C

then restarted the ranger and it was successful.

again facing the same issue:


build detail:  
ambari-agent-2.4.0.0-625.x86_64


Diffs
-

  ambari-agent/src/main/python/ambari_agent/main.py 32e522d 

Diff: https://reviews.apache.org/r/48281/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Re: Review Request 48044: Provide context for hdp-select failures during ambari component install

[Laszlo Puskas]

> On June 2, 2016, 4:23 p.m., Alejandro Fernandez wrote:
> > ambari-common/src/main/python/resource_management/libraries/functions/hdp_select.py,
> >  line 129
> > 
> >
> > This cannot assume the directory will be /usr/hdp, instead use 
> > stack_root variable.
> 
> Sumit Mohanty wrote:
> Looks like different patches are needed for branch-2.4 and branch-2.2. 
> Starting 2.4 {stack_root}/{version_to_select} can be used and for the older 
> version it can be /usr/hdp.

Using the stack_root to the log commands.


- Laszlo


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48044/#review135943
---


On June 6, 2016, 2:48 p.m., Laszlo Puskas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48044/
> ---
> 
> (Updated June 6, 2016, 2:48 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Oliver Szabo, Sumit Mohanty, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-16952
> https://issues.apache.org/jira/browse/AMBARI-16952
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In case the hdp-select command fails during a service / component 
> installation there's no contextual information about the cause of the failure.
> This issue is for logging information about the machine on which the 
> hdp-select command fails.
> This solution wraps hdp-select command calls in a try/catch block and logs 
> failure / hdp installationrelated information.
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/hdp_select.py
>  9a3201e 
> 
> Diff: https://reviews.apache.org/r/48044/diff/
> 
> 
> Testing
> ---
> 
> Manual testing underway.
> 
> 
> Thanks,
> 
> Laszlo Puskas
> 
>

Re: Review Request 48044: Provide context for hdp-select failures during ambari component install

[Laszlo Puskas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48044/
---

(Updated June 6, 2016, 2:48 p.m.)


Review request for Ambari, Andrew Onischuk, Oliver Szabo, Sumit Mohanty, and 
Sebastian Toader.


Changes
---

Added stack_root variable


Bugs: AMBARI-16952
https://issues.apache.org/jira/browse/AMBARI-16952


Repository: ambari


Description
---

In case the hdp-select command fails during a service / component installation 
there's no contextual information about the cause of the failure.
This issue is for logging information about the machine on which the hdp-select 
command fails.
This solution wraps hdp-select command calls in a try/catch block and logs 
failure / hdp installationrelated information.


Diffs (updated)
-

  
ambari-common/src/main/python/resource_management/libraries/functions/hdp_select.py
 9a3201e 

Diff: https://reviews.apache.org/r/48044/diff/


Testing
---

Manual testing underway.


Thanks,

Laszlo Puskas

Re: Review Request 48278: Service stop/start fail after Ambari upgrade due to missing config parameters

[Dmitro Lisnichenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48278/#review136256
---


Ship it!




Ship It!

- Dmitro Lisnichenko


On June 6, 2016, 5:14 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48278/
> ---
> 
> (Updated June 6, 2016, 5:14 p.m.)
> 
> 
> Review request for Ambari and Dmitro Lisnichenko.
> 
> 
> Bugs: AMBARI-17059
> https://issues.apache.org/jira/browse/AMBARI-17059
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> ambari-server --hash  
> 43b564767ccf9eaa60f4e352c46ed3cb32dc7e78
> 
> 
> Observed this issue in a couple of scenarios:
> 
> **Scenario #1**
> 
>   1. Deploy HDP-2.4.2.0 with Ambari 2.2.2.0
>   2. Upgrade Ambari to 2.4.0.0-635 build
>   3. Register HDP-2.5.0.0-639 and install the bits
>   4. Start Express Upgrade to 2.5
> 
> Oozie server start failed with below error:
> 
> 
> 
> 
> Traceback (most recent call last):
>   File 
> "/var/lib/ambari-agent/cache/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server.py",
>  line 215, in 
> OozieServer().execute()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 257, in execute
> method(env)
>   File 
> "/var/lib/ambari-agent/cache/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server.py",
>  line 100, in stop
> oozie_service(action='stop', upgrade_type=upgrade_type)
>   File 
> "/usr/lib/python2.6/site-packages/ambari_commons/os_family_impl.py", line 89, 
> in thunk
> return fn(*args, **kwargs)
>   File 
> "/var/lib/ambari-agent/cache/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_service.py",
>  line 149, in oozie_service
> stop_cmd  = format("cd {oozie_tmp_dir} && 
> {oozie_home}/bin/oozie-stop.sh")
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/format.py",
>  line 95, in format
> return ConfigurationFormatter().format(format_string, args, **result)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/format.py",
>  line 59, in format
> result_protected = self.vformat(format_string, args, all_params)
>   File "/usr/lib64/python2.7/string.py", line 549, in vformat
> result = self._vformat(format_string, args, kwargs, used_args, 2)
>   File "/usr/lib64/python2.7/string.py", line 582, in _vformat
> result.append(self.format_field(obj, format_spec))
>   File "/usr/lib64/python2.7/string.py", line 599, in format_field
> return format(value, format_spec)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/config_dictionary.py",
>  line 73, in __getattr__
> raise Fail("Configuration parameter '" + self.name + "' was not found 
> in configurations dictionary!")
> resource_management.core.exceptions.Fail: Configuration parameter 
> 'oozie_tmp_dir' was not found in configurations dictionary!
> 
> 
> **Scenario #2**
> 
>   1. Deploy HDP-2.3.4.0 with Ambari 2.2.0.0
>   2. Upgrade Ambari to 2.4.0.0-635 build
>   3. Delete Ranger service
>   4. Restart all services with stale configs
> 
> **Result**  
> ATS, DN, Falcon client restart fails with below error
> 
> 
> 
> 
> Traceback (most recent call last):
>   File 
> "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/hook.py",
>  line 39, in 
> BeforeStartHook().execute()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 257, in execute
> method(env)
>   File 
> "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/hook.py",
>  line 28, in hook
> import params
>   File 
> "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/params.py",
>  line 316, in 
> immutable_paths = get_not_managed_resources(),
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/get_not_managed_resources.py",
>  line 38, in get_not_managed_resources
> managed_hdfs_resource_property_list = filter(None, [property.strip() 
> for property in managed_hdfs_resource_property_names.split(',')])
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/config_dictionary.py",
>  line 73, in __getattr__
> raise Fail("Configuration parameter '" + self.name + "' was not found 
> in configurations dictionary!")
> resource_management.core.exceptions.Fail: Configuration parameter 
> 'managed_hdfs_resource_property_names' was not found in configurations 
> dictionary!
> Error: Error: Unable to 

Review Request 48279: Visual explain unit tests

[Pallav Kulshreshtha]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48279/
---

Review request for Ambari, DIPAYAN BHOWMICK, Gaurav Nagar, and Nitiraj Rathore.


Bugs: AMBARI-17060
https://issues.apache.org/jira/browse/AMBARI-17060


Repository: ambari


Description
---

- test cases written for visual explain


Diffs
-

  contrib/views/hive/src/main/resources/ui/hive-web/app/templates/databases.hbs 
290cdac 
  contrib/views/hive/src/main/resources/ui/hive-web/tests/helpers/api-mock.js 
ed4822d 
  contrib/views/hive/src/main/resources/ui/hive-web/tests/helpers/dbclick.js 
PRE-CREATION 
  
contrib/views/hive/src/main/resources/ui/hive-web/tests/integration/database-test.js
 52cda77 
  
contrib/views/hive/src/main/resources/ui/hive-web/tests/integration/query-editor-test.js
 b409e12 
  
contrib/views/hive/src/main/resources/ui/hive-web/tests/integration/saved-queries-test.js
 c444523 
  
contrib/views/hive/src/main/resources/ui/hive-web/tests/integration/udfs-test.js
 95a0043 
  
contrib/views/hive/src/main/resources/ui/hive-web/tests/unit/components/udf-tr-view-test.js
 PRE-CREATION 
  
contrib/views/hive/src/main/resources/ui/hive-web/tests/unit/controllers/udfs-test.js
 5bd369e 

Diff: https://reviews.apache.org/r/48279/diff/


Testing
---

ember test --server


Thanks,

Pallav Kulshreshtha

Review Request 48278: Service stop/start fail after Ambari upgrade due to missing config parameters

[Andrew Onischuk]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48278/
---

Review request for Ambari and Dmitro Lisnichenko.


Bugs: AMBARI-17059
https://issues.apache.org/jira/browse/AMBARI-17059


Repository: ambari


Description
---

ambari-server --hash  
43b564767ccf9eaa60f4e352c46ed3cb32dc7e78


Observed this issue in a couple of scenarios:

**Scenario #1**

  1. Deploy HDP-2.4.2.0 with Ambari 2.2.2.0
  2. Upgrade Ambari to 2.4.0.0-635 build
  3. Register HDP-2.5.0.0-639 and install the bits
  4. Start Express Upgrade to 2.5

Oozie server start failed with below error:




Traceback (most recent call last):
  File 
"/var/lib/ambari-agent/cache/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server.py",
 line 215, in 
OozieServer().execute()
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
 line 257, in execute
method(env)
  File 
"/var/lib/ambari-agent/cache/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server.py",
 line 100, in stop
oozie_service(action='stop', upgrade_type=upgrade_type)
  File "/usr/lib/python2.6/site-packages/ambari_commons/os_family_impl.py", 
line 89, in thunk
return fn(*args, **kwargs)
  File 
"/var/lib/ambari-agent/cache/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_service.py",
 line 149, in oozie_service
stop_cmd  = format("cd {oozie_tmp_dir} && 
{oozie_home}/bin/oozie-stop.sh")
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/functions/format.py",
 line 95, in format
return ConfigurationFormatter().format(format_string, args, **result)
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/functions/format.py",
 line 59, in format
result_protected = self.vformat(format_string, args, all_params)
  File "/usr/lib64/python2.7/string.py", line 549, in vformat
result = self._vformat(format_string, args, kwargs, used_args, 2)
  File "/usr/lib64/python2.7/string.py", line 582, in _vformat
result.append(self.format_field(obj, format_spec))
  File "/usr/lib64/python2.7/string.py", line 599, in format_field
return format(value, format_spec)
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/config_dictionary.py",
 line 73, in __getattr__
raise Fail("Configuration parameter '" + self.name + "' was not found 
in configurations dictionary!")
resource_management.core.exceptions.Fail: Configuration parameter 
'oozie_tmp_dir' was not found in configurations dictionary!


**Scenario #2**

  1. Deploy HDP-2.3.4.0 with Ambari 2.2.0.0
  2. Upgrade Ambari to 2.4.0.0-635 build
  3. Delete Ranger service
  4. Restart all services with stale configs

**Result**  
ATS, DN, Falcon client restart fails with below error




Traceback (most recent call last):
  File 
"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/hook.py",
 line 39, in 
BeforeStartHook().execute()
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
 line 257, in execute
method(env)
  File 
"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/hook.py",
 line 28, in hook
import params
  File 
"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/params.py",
 line 316, in 
immutable_paths = get_not_managed_resources(),
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/functions/get_not_managed_resources.py",
 line 38, in get_not_managed_resources
managed_hdfs_resource_property_list = filter(None, [property.strip() 
for property in managed_hdfs_resource_property_names.split(',')])
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/config_dictionary.py",
 line 73, in __getattr__
raise Fail("Configuration parameter '" + self.name + "' was not found 
in configurations dictionary!")
resource_management.core.exceptions.Fail: Configuration parameter 
'managed_hdfs_resource_property_names' was not found in configurations 
dictionary!
Error: Error: Unable to run the custom hook script ['/usr/bin/python', 
'/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/hook.py',
 'START', '/var/lib/ambari-agent/data/command-603.json', 
'/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START', 
'/var/lib/ambari-agent/data/structured-out-603.json', 'INFO', 
'/var/lib/ambari-agent/tmp']


Diffs
-

  
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/configuration/atlas-env.xml
 114a0fa 
  
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/configuration/falcon-env.xml
 5f607e3 
  

Re: Review Request 48276: View config-Custom views are not working after kerberization

[Nitiraj Rathore]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48276/#review136249
---


Ship it!




Ship It!

- Nitiraj Rathore


On June 6, 2016, 12:47 p.m., Gaurav Nagar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48276/
> ---
> 
> (Updated June 6, 2016, 12:47 p.m.)
> 
> 
> Review request for Ambari, DIPAYAN BHOWMICK, Nitiraj Rathore, Pallav 
> Kulshreshtha, Rohit Choudhary, and Ashwin Rajeev.
> 
> 
> Bugs: AMBARI-17056
> https://issues.apache.org/jira/browse/AMBARI-17056
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Fixed setting of webhdfs auth for custom views.
> 
> 
> Diffs
> -
> 
>   
> contrib/views/hive/src/main/java/org/apache/ambari/view/hive/client/ConnectionFactory.java
>  0eb7906 
>   
> contrib/views/utils/src/main/java/org/apache/ambari/view/utils/hdfs/AuthConfigurationBuilder.java
>  10ec1fb 
>   
> contrib/views/utils/src/main/java/org/apache/ambari/view/utils/hdfs/ConfigurationBuilder.java
>  9edf9d7 
> 
> Diff: https://reviews.apache.org/r/48276/diff/
> 
> 
> Testing
> ---
> 
> Manually tested
> 
> 
> Thanks,
> 
> Gaurav Nagar
> 
>

Re: Review Request 48266: Add explicit ambari-server log line indicating cluster creation complete

[Laszlo Puskas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48266/#review136244
---




ambari-server/src/main/java/org/apache/ambari/server/topology/TopologyManager.java
 (line 162)


I'd rather perform this check out of this method, in order to always have a 
boolean return value for this method.


- Laszlo Puskas


On June 6, 2016, 8:47 a.m., Daniel Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48266/
> ---
> 
> (Updated June 6, 2016, 8:47 a.m.)
> 
> 
> Review request for Ambari, Laszlo Puskas, Oliver Szabo, Robert Nettleton, 
> Sandor Magyari, Sumit Mohanty, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-17053
> https://issues.apache.org/jira/browse/AMBARI-17053
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add a log message to see when the cluster is ready to use (cluster creation 
> finishes).
> 
> An event after each heartbeat. At that time cluster provision request is 
> checked if it is finished or not. In case of an ambari server restart, the 
> replayed requests are used to determine which one was the provision request. 
> I could not find a nice way to do it, but I could make a workaround.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/HeartbeatProcessor.java
>  c6036c2 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ProvisionClusterRequest.java
>  3feac55 
>   
> ambari-server/src/main/java/org/apache/ambari/server/events/AmbariEvent.java 
> 1079806 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/TopologyManager.java
>  e3f5b49 
>   
> ambari-server/src/test/java/org/apache/ambari/server/topology/TopologyManagerTest.java
>  fd8653c 
> 
> Diff: https://reviews.apache.org/r/48266/diff/
> 
> 
> Testing
> ---
> 
> Still running locally...
> 
> 
> Thanks,
> 
> Daniel Gergely
> 
>

Review Request 48276: View config-Custom views are not working after kerberization

[Gaurav Nagar]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48276/
---

Review request for Ambari, DIPAYAN BHOWMICK, Nitiraj Rathore, Pallav 
Kulshreshtha, Rohit Choudhary, and Ashwin Rajeev.


Bugs: AMBARI-17056
https://issues.apache.org/jira/browse/AMBARI-17056


Repository: ambari


Description
---

Fixed setting of webhdfs auth for custom views.


Diffs
-

  
contrib/views/hive/src/main/java/org/apache/ambari/view/hive/client/ConnectionFactory.java
 0eb7906 
  
contrib/views/utils/src/main/java/org/apache/ambari/view/utils/hdfs/AuthConfigurationBuilder.java
 10ec1fb 
  
contrib/views/utils/src/main/java/org/apache/ambari/view/utils/hdfs/ConfigurationBuilder.java
 9edf9d7 

Diff: https://reviews.apache.org/r/48276/diff/


Testing
---

Manually tested


Thanks,

Gaurav Nagar

Re: Review Request 48229: Refactor service_advisor apis to remove passing of stack_advisor

[Tim Thorpe]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48229/#review136240
---




ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py 
(line 40)


This should be removed in the final code



ambari-server/src/main/resources/stacks/service_advisor.py (line 51)


I originally had implemented the service advisor like this.  I had modified 
it specifically NOT to inherit from the DefaultStackAdvisor after comments 
suggesting that the inheritance didn't make sense.

The way I see it both the stack advisor and service advisor should inherit 
the core functions from a generic advisor but this implementation is the 
easiest.



ambari-server/src/main/resources/stacks/stack_advisor.py (line 412)


This is really not the method, it is the advisor.



ambari-server/src/main/resources/stacks/stack_advisor.py (line 434)


This is really the advisor not the method.


- Tim Thorpe


On June 5, 2016, 10:18 p.m., Lav Jain wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48229/
> ---
> 
> (Updated June 5, 2016, 10:18 p.m.)
> 
> 
> Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, Jayush 
> Luniya, Matt, and Tim Thorpe.
> 
> 
> Bugs: AMBARI-16663
> https://issues.apache.org/jira/browse/AMBARI-16663
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> The original intent was to move the common functions that are used by both 
> stack and service advisors to a separate utils file to avoid passing 
> stack_advisor class into service advisor apis.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py
>  28eb82f 
>   
> ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py 
> 9b34171 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
> 1dedf3d 
>   ambari-server/src/main/resources/stacks/service_advisor.py 3d6c293 
>   ambari-server/src/main/resources/stacks/stack_advisor.py beba225 
>   ambari-server/src/test/python/stacks/2.3/HAWQ/test_service_advisor.py 
> 5750938 
>   ambari-server/src/test/python/stacks/2.3/PXF/test_service_advisor.py 
> c3a63cc 
>   ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 
> 424a386 
> 
> Diff: https://reviews.apache.org/r/48229/diff/
> 
> 
> Testing
> ---
> 
> Lavs-MacBook-Pro:common ljain$ python -m discover -v
> test_createComponentLayoutRecommendations_hawq_1_Host 
> (test_stack_advisor.TestHDP23StackAdvisor) ... ServiceAdvisor implementation 
> for service HAWQ was loaded
> [u'c6401.ambari.apache.org']
> [u'c6401.ambari.apache.org']
> ok
> test_createComponentLayoutRecommendations_hawq_3_Hosts 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test that HAWQSTANDBY is recommended on a 3-node cluster ... ServiceAdvisor 
> implementation for service HAWQ was loaded
> [u'c6401.ambari.apache.org', u'c6402.ambari.apache.org', 
> u'c6403.ambari.apache.org']
> [u'c6401.ambari.apache.org', u'c6402.ambari.apache.org', 
> u'c6403.ambari.apache.org']
> ok
> test_createComponentLayoutRecommendations_hawqsegment_add_service_wizard_already_installed
>  (test_stack_advisor.TestHDP23StackAdvisor)
> Test that HAWQSEGMENT does not get recommended during Add Service Wizard, 
> when HAWQ has already been installed ... ServiceAdvisor implementation for 
> service HAWQ was loaded
> ok
> test_createComponentLayoutRecommendations_hawqsegment_add_service_wizard_to_be_installed
>  (test_stack_advisor.TestHDP23StackAdvisor)
> Test that HAWQSEGMENT gets recommended correctly during Add Service Wizard, 
> when HAWQ is selected for installation ... ServiceAdvisor implementation for 
> service HAWQ was loaded
> ok
> test_createComponentLayoutRecommendations_hawqsegment_cluster_install 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test that HAWQSEGMENT gets recommended correctly during Cluster Install 
> Wizard, when HAWQ is selected for installation ... ServiceAdvisor 
> implementation for service HAWQ was loaded
> ok
> test_createComponentLayoutRecommendations_no_hawq_3_Hosts 
> (test_stack_advisor.TestHDP23StackAdvisor)
> Test no failures when there are no HAWQ components ... ok
> test_createComponentLayoutRecommendations_pxf_add_service_wizard_already_installed
>  (test_stack_advisor.TestHDP23StackAdvisor)
> Test that PXF does not get recommended during Add Service Wizard, when PXF 
> has already been installed ... ServiceAdvisor implementation for service PXF 
> was loaded
> ok
> 

Review Request 48274: Show better error message when VIEW USER can not access CAPACITY SCHEDULER View

[Gaurav Nagar]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48274/
---

Review request for Ambari, DIPAYAN BHOWMICK, Nitiraj Rathore, Pallav 
Kulshreshtha, Rohit Choudhary, and Ashwin Rajeev.


Bugs: AMBARI-17055
https://issues.apache.org/jira/browse/AMBARI-17055


Repository: ambari


Description
---

Added error message when user does not have read permission on configurations.
Added message when view is in read only mode.


Diffs
-

  
contrib/views/capacity-scheduler/src/main/java/org/apache/ambari/view/capacityscheduler/ConfigurationService.java
 42076fe 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/queues.hbs 
94cb421 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/queues/index.hbs
 1c468b5 
  
contrib/views/utils/src/main/java/org/apache/ambari/view/utils/ambari/AmbariApi.java
 86d8434 

Diff: https://reviews.apache.org/r/48274/diff/


Testing
---

Manully Tested


Thanks,

Gaurav Nagar

Re: Review Request 48234: Falcon server fails to start, HDP 2.4 to use data-mirroring directory, HDP 2.5 to use extensions

[Di Li]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48234/#review136242
---


Ship it!




Ship It!

- Di Li


On June 3, 2016, 10:47 p.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48234/
> ---
> 
> (Updated June 3, 2016, 10:47 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Di Li, Dmitro Lisnichenko, 
> Jonathan Hurley, Jayush Luniya, Nate Cole, and Tim Thorpe.
> 
> 
> Bugs: AMBARI-17037
> https://issues.apache.org/jira/browse/AMBARI-17037
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Fresh installation and EU/RU of Falcon is failing.
> In HDP 2.4, Falcon used the data mirroring folder.
> In HDP 2.5, data mirroring is no longer used and instead uses the extensions 
> folder that must be uploaded to HDFS.
> 
> This means that EU/RU from HDP 2.4 to 2.5 must also upload the extensions 
> folder to HDFS
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/constants.py
>  34d1a1a 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py
>  4c13aaa 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon.py
>  5e25325 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon_server.py
>  7c2cdf4 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/params_linux.py
>  8c2ad8e 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_features.json
>  8b669e8 
>   ambari-server/src/test/python/stacks/2.0.6/configs/default.json 8d04e36 
> 
> Diff: https://reviews.apache.org/r/48234/diff/
> 
> 
> Testing
> ---
> 
> Python unit tests passed.
> 
> --
> Total run:1052
> Total errors:0
> Total failures:0
> OK
> 
> Express Upgrade is currently broken, so will commit this patch and retest 
> once it is stable again in Hadoop Core.
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Re: Review Request 48234: Falcon server fails to start, HDP 2.4 to use data-mirroring directory, HDP 2.5 to use extensions

[Tim Thorpe]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48234/#review136238
---


Ship it!




Ship It!

- Tim Thorpe


On June 3, 2016, 10:47 p.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48234/
> ---
> 
> (Updated June 3, 2016, 10:47 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Di Li, Dmitro Lisnichenko, 
> Jonathan Hurley, Jayush Luniya, Nate Cole, and Tim Thorpe.
> 
> 
> Bugs: AMBARI-17037
> https://issues.apache.org/jira/browse/AMBARI-17037
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Fresh installation and EU/RU of Falcon is failing.
> In HDP 2.4, Falcon used the data mirroring folder.
> In HDP 2.5, data mirroring is no longer used and instead uses the extensions 
> folder that must be uploaded to HDFS.
> 
> This means that EU/RU from HDP 2.4 to 2.5 must also upload the extensions 
> folder to HDFS
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/constants.py
>  34d1a1a 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py
>  4c13aaa 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon.py
>  5e25325 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon_server.py
>  7c2cdf4 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/params_linux.py
>  8c2ad8e 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_features.json
>  8b669e8 
>   ambari-server/src/test/python/stacks/2.0.6/configs/default.json 8d04e36 
> 
> Diff: https://reviews.apache.org/r/48234/diff/
> 
> 
> Testing
> ---
> 
> Python unit tests passed.
> 
> --
> Total run:1052
> Total errors:0
> Total failures:0
> OK
> 
> Express Upgrade is currently broken, so will commit this patch and retest 
> once it is stable again in Hadoop Core.
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Re: Review Request 48265: Comparison tab : Allow the comparison only when minimum 2 tabs are selected and minor fixes

[Miklos Gergely]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48265/#review136237
---


Ship it!




Ship It!

- Miklos Gergely


On June 6, 2016, 10:42 a.m., Dharmesh Makwana wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48265/
> ---
> 
> (Updated June 6, 2016, 10:42 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Andrew Onischuk, Don Bosco 
> Durai, Jaimin Jetly, Oliver Szabo, Robert Nettleton, Sandor Magyari, Sumit 
> Mohanty, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-17052
> https://issues.apache.org/jira/browse/AMBARI-17052
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Patch contains
> 1) When only one tab is selected for comparison then show message to user 
> saying minimum 2 tabs should be selected for comparison.
> 2) Minor fixes.
> 
> 
> Diffs
> -
> 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/views/dashboard/MainLayoutView.js
>  c2e0bb3 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/views/graphs/GraphLayoutView.js
>  79a3a02 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/views/tabs/HierarchyTabLayoutView.js
>  3290216 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/webapp/templates/tabs/HierarchyTabLayoutView_tmpl.html
>  1a3135f 
> 
> Diff: https://reviews.apache.org/r/48265/diff/
> 
> 
> Testing
> ---
> 
> Setup logsearch 2 node cluster and tested the above fixes.
> 
> 
> Thanks,
> 
> Dharmesh Makwana
> 
>

Re: Review Request 48265: Comparison tab : Allow the comparison only when minimum 2 tabs are selected and minor fixes

[Oliver Szabo]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48265/#review136235
---


Ship it!




Ship It!

- Oliver Szabo


On June 6, 2016, 10:42 a.m., Dharmesh Makwana wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48265/
> ---
> 
> (Updated June 6, 2016, 10:42 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Andrew Onischuk, Don Bosco 
> Durai, Jaimin Jetly, Oliver Szabo, Robert Nettleton, Sandor Magyari, Sumit 
> Mohanty, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-17052
> https://issues.apache.org/jira/browse/AMBARI-17052
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Patch contains
> 1) When only one tab is selected for comparison then show message to user 
> saying minimum 2 tabs should be selected for comparison.
> 2) Minor fixes.
> 
> 
> Diffs
> -
> 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/views/dashboard/MainLayoutView.js
>  c2e0bb3 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/views/graphs/GraphLayoutView.js
>  79a3a02 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/views/tabs/HierarchyTabLayoutView.js
>  3290216 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/webapp/templates/tabs/HierarchyTabLayoutView_tmpl.html
>  1a3135f 
> 
> Diff: https://reviews.apache.org/r/48265/diff/
> 
> 
> Testing
> ---
> 
> Setup logsearch 2 node cluster and tested the above fixes.
> 
> 
> Thanks,
> 
> Dharmesh Makwana
> 
>

Re: Review Request 48212: Fix files mentioned by ServicePropertiesTest on latest commits

[Dmitro Lisnichenko]

> On June 3, 2016, 8:16 p.m., Jayush Luniya wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-conf.xml,
> >  line 32
> > 
> >
> > Why are these mandatory property attributes to cause UT failures? 
> > Shouldnt they have defaults if not defined?
> > 
> > Also how do we decide what to set these properties too? (i.e. 
> > on-ambari-upgrade add="false" v/s on-ambari-upgrade add="true") Can you 
> > provide examples? 
> > 
> > I am not clear on why on-ambari-upgrade delete="true" and 
> > on-stack-upgrade delete="false"?
> > 
> > I will look at your original patch to get some context and then review 
> > later today.

the idea is to force developers to specify what to do during upgrade explicitly 
when adding properties, instead of trying to guess correct policy during upgrade
Regarding documentation - as of now patch causes troubles during RU/EU, and may 
be partially reverted soon. So let's return to this question later


- Dmitro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48212/#review136078
---


On June 3, 2016, 7:55 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48212/
> ---
> 
> (Updated June 3, 2016, 7:55 p.m.)
> 
> 
> Review request for Ambari and Jayush Luniya.
> 
> 
> Bugs: AMBARI-17032
> https://issues.apache.org/jira/browse/AMBARI-17032
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> {code}
> ServicePropertiesTest.validatePropertySchemaOfServiceXMLs:50 » Ambari File 
> /tm...
> {code}
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-conf.xml
>  d0acdda 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-env.xml
>  410a1c1 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-log4j-properties.xml
>  d84207a 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-spark-blacklist.xml
>  4a5fbfb 
> 
> Diff: https://reviews.apache.org/r/48212/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Re: Review Request 48265: Comparison tab : Allow the comparison only when minimum 2 tabs are selected and minor fixes

[Dharmesh Makwana]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48265/
---

(Updated June 6, 2016, 10:42 a.m.)


Review request for Ambari, Alejandro Fernandez, Andrew Onischuk, Don Bosco 
Durai, Jaimin Jetly, Oliver Szabo, Robert Nettleton, Sandor Magyari, Sumit 
Mohanty, and Sebastian Toader.


Changes
---

Fixed review comments.


Bugs: AMBARI-17052
https://issues.apache.org/jira/browse/AMBARI-17052


Repository: ambari


Description
---

Patch contains
1) When only one tab is selected for comparison then show message to user 
saying minimum 2 tabs should be selected for comparison.
2) Minor fixes.


Diffs (updated)
-

  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/views/dashboard/MainLayoutView.js
 c2e0bb3 
  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/views/graphs/GraphLayoutView.js
 79a3a02 
  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/views/tabs/HierarchyTabLayoutView.js
 3290216 
  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/templates/tabs/HierarchyTabLayoutView_tmpl.html
 1a3135f 

Diff: https://reviews.apache.org/r/48265/diff/


Testing
---

Setup logsearch 2 node cluster and tested the above fixes.


Thanks,

Dharmesh Makwana

Re: Review Request 48265: Comparison tab : Allow the comparison only when minimum 2 tabs are selected and minor fixes

[Oliver Szabo]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48265/#review136231
---




ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/views/tabs/HierarchyTabLayoutView.js
 (line 149)


Remove this line instead of commenting it.



ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/views/tabs/HierarchyTabLayoutView.js
 (line 165)


same as above


- Oliver Szabo


On June 6, 2016, 8:39 a.m., Dharmesh Makwana wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48265/
> ---
> 
> (Updated June 6, 2016, 8:39 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Andrew Onischuk, Don Bosco 
> Durai, Jaimin Jetly, Oliver Szabo, Robert Nettleton, Sandor Magyari, Sumit 
> Mohanty, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-17052
> https://issues.apache.org/jira/browse/AMBARI-17052
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Patch contains
> 1) When only one tab is selected for comparison then show message to user 
> saying minimum 2 tabs should be selected for comparison.
> 2) Minor fixes.
> 
> 
> Diffs
> -
> 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/views/dashboard/MainLayoutView.js
>  c2e0bb3 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/views/graphs/GraphLayoutView.js
>  79a3a02 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/views/tabs/HierarchyTabLayoutView.js
>  3290216 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/webapp/templates/tabs/HierarchyTabLayoutView_tmpl.html
>  1a3135f 
> 
> Diff: https://reviews.apache.org/r/48265/diff/
> 
> 
> Testing
> ---
> 
> Setup logsearch 2 node cluster and tested the above fixes.
> 
> 
> Thanks,
> 
> Dharmesh Makwana
> 
>

Review Request 48266: Add explicit ambari-server log line indicating cluster creation complete

[Daniel Gergely]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48266/
---

Review request for Ambari, Laszlo Puskas, Oliver Szabo, Robert Nettleton, 
Sandor Magyari, Sumit Mohanty, and Sebastian Toader.


Bugs: AMBARI-17053
https://issues.apache.org/jira/browse/AMBARI-17053


Repository: ambari


Description
---

Add a log message to see when the cluster is ready to use (cluster creation 
finishes).

An event after each heartbeat. At that time cluster provision request is 
checked if it is finished or not. In case of an ambari server restart, the 
replayed requests are used to determine which one was the provision request. I 
could not find a nice way to do it, but I could make a workaround.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/agent/HeartbeatProcessor.java
 c6036c2 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ProvisionClusterRequest.java
 3feac55 
  ambari-server/src/main/java/org/apache/ambari/server/events/AmbariEvent.java 
1079806 
  
ambari-server/src/main/java/org/apache/ambari/server/topology/TopologyManager.java
 e3f5b49 
  
ambari-server/src/test/java/org/apache/ambari/server/topology/TopologyManagerTest.java
 fd8653c 

Diff: https://reviews.apache.org/r/48266/diff/


Testing
---

Still running locally...


Thanks,

Daniel Gergely

Review Request 48265: Comparison tab : Allow the comparison only when minimum 2 tabs are selected and minor fixes

[Dharmesh Makwana]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48265/
---

Review request for Ambari, Alejandro Fernandez, Andrew Onischuk, Don Bosco 
Durai, Jaimin Jetly, Oliver Szabo, Robert Nettleton, Sandor Magyari, Sumit 
Mohanty, and Sebastian Toader.


Bugs: AMBARI-17052
https://issues.apache.org/jira/browse/AMBARI-17052


Repository: ambari


Description
---

Patch contains
1) When only one tab is selected for comparison then show message to user 
saying minimum 2 tabs should be selected for comparison.
2) Minor fixes.


Diffs
-

  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/views/dashboard/MainLayoutView.js
 c2e0bb3 
  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/views/graphs/GraphLayoutView.js
 79a3a02 
  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/views/tabs/HierarchyTabLayoutView.js
 3290216 
  
ambari-logsearch/ambari-logsearch-portal/src/main/webapp/templates/tabs/HierarchyTabLayoutView_tmpl.html
 1a3135f 

Diff: https://reviews.apache.org/r/48265/diff/


Testing
---

Setup logsearch 2 node cluster and tested the above fixes.


Thanks,

Dharmesh Makwana

Review Request 48262: AMBARI-16852: Update metrics collector pom.xml to use httpclient-4.2.5.jar

[Qin Liu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48262/
---

Review request for Ambari, Aravindan Vijayan, Dmytro Sen, and Sid Wagle.


Bugs: AMBARI-16852
https://issues.apache.org/jira/browse/AMBARI-16852


Repository: ambari


Description
---

httpclient-4.0.1.jar used by Ambari Metrics collector does not have method 
URLEncodedUtils.parse (final URI uri, final String encoding). This will cause 
the metrics not to be populated and the collector log to be flooded with the 
following exception when the Ambari Metrics service is in distributed mode and 
the user configures http authentication filter without enabling Kerberos:

21:54:08,215 ERROR [1831490216@qtp-1883237069-7582] log:87 - Error for 
/ws/v1/timeline/metrics
java.lang.NoSuchMethodError: 
org.apache.http.client.utils.URLEncodedUtils.parse(Ljava/lang/String;Ljava/nio/charset/Charset;)Ljava/util/List;
at 
org.apache.hadoop.security.authentication.server.PseudoAuthenticationHandler.getUserName(PseudoAuthenticationHandler.java:143)
at 
org.apache.hadoop.security.authentication.server.PseudoAuthenticationHandler.authenticate(PseudoAuthenticationHandler.java:179)
at 
org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:507)
at 
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at 
org.apache.hadoop.http.HttpServer2$QuotingInputFilter.doFilter(HttpServer2.java:1224)
at 
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
at 
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)

Steps to reproduce:
1. install a cluster with default from Ambari Web UI.
2. configure Ambari Metrics service in distributed mode.
3. Add the following properties in Custom core-site from Ambari Web UI:
1). hadoop.http.authentication.signature.secret.file=/etc/security/http_secret.
2). 
hadoop.http.filter.initializers=org.apache.hadoop.security.AuthenticationFilterInitialize.
4. create /etc/security/http_secret text file on each node.
5. restart HDFS, YARN, Mapreduce2, HBase, Kafka, and Ambari Metrics service.
6. Metrics on dashboard and HDFS, YARN, HBase, Kafka, and Amabari Metrics will 
show No Data Available.


Diffs
-

  ambari-metrics/ambari-metrics-timelineservice/pom.xml 29f7812 

Diff: https://reviews.apache.org/r/48262/diff/


Testing
---

FIX:
1. update Ambari Metrics collector pom.xml to use httpclient-4.2.5.jar.
2. without the fix, the collector lib contains httpclient-4.0.1.jar and 
httpcore-4.2.5.jar.
3. with the fix, the collector lib will have httpclient-4.2.5.jar and 
httpcore-4.2.4.jar.

TESTS:
1. The Ambari Metrics unit tests.
2. Also tested with HDP UI - after replacing httpclient-4.0.1.jar and 
httpcore-4.2.5.jar with httpclient-4.2.5.jar and httpcore-4.2.4.jar, the 
missing metrics show up.


Thanks,

Qin Liu