date:20161108


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53592/#review155394
---



This is a massive code review. Any comments would be useful to navigate this.

- Alejandro Fernandez


On Nov. 9, 2016, 1:36 a.m., Sumit Mohanty wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53592/
> ---
> 
> (Updated Nov. 9, 2016, 1:36 a.m.)
> 
> 
> Review request for Ambari, Sriharsha Chintalapani and Sid Wagle.
> 
> 
> Bugs: AMBARI-18829
> https://issues.apache.org/jira/browse/AMBARI-18829
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Allow Ambari to manage Kafka's Custom JAAS Config.
> Move the j2 templates into config properties.
> 
> I intentionally left the j2 templates and logic to fall back to j2 template 
> for now. They can be removed later.
> 
> 
> Diffs
> -
> 
>   LICENSE.txt f05016f 
>   ambari-admin/pom.xml 4a8bba3 
>   ambari-admin/src/main/resources/ui/admin-web/app/index.html b3f1bd1 
>   ambari-admin/src/main/resources/ui/admin-web/app/scripts/app.js 80e2813 
>   
> ambari-admin/src/main/resources/ui/admin-web/app/scripts/controllers/ambariViews/CreateViewInstanceCtrl.js
>  94b8cc1 
>   
> ambari-admin/src/main/resources/ui/admin-web/app/scripts/controllers/clusters/ExportBlueprintCtrl.js
>  a1e4fa0 
>   
> ambari-admin/src/main/resources/ui/admin-web/app/scripts/controllers/mainCtrl.js
>  c945644 
>   
> ambari-admin/src/main/resources/ui/admin-web/app/scripts/controllers/users/UsersShowCtrl.js
>  f7ed7ee 
>   ambari-admin/src/main/resources/ui/admin-web/app/scripts/i18n.config.js 
> cd9b922 
>   ambari-admin/src/main/resources/ui/admin-web/app/scripts/routes.js a1ca59c 
>   
> ambari-admin/src/main/resources/ui/admin-web/app/scripts/services/Cluster.js 
> 02c231a 
>   
> ambari-admin/src/main/resources/ui/admin-web/app/scripts/services/PermissionLoader.js
>  9cc04e4 
>   
> ambari-admin/src/main/resources/ui/admin-web/app/scripts/services/Utility.js 
> 7ff10f8 
>   
> ambari-admin/src/main/resources/ui/admin-web/app/views/ambariViews/create.html
>  25f1487 
>   
> ambari-admin/src/main/resources/ui/admin-web/app/views/ambariViews/listTable.html
>  e03f035 
>   
> ambari-admin/src/main/resources/ui/admin-web/app/views/clusters/exportBlueprint.html
>  6ccbffe 
>   ambari-admin/src/main/resources/ui/admin-web/app/views/groups/edit.html 
> 78a6884 
>   ambari-admin/src/main/resources/ui/admin-web/app/views/leftNavbar.html 
> a7fc18a 
>   ambari-admin/src/main/resources/ui/admin-web/app/views/users/create.html 
> cc5d8d4 
>   
> ambari-admin/src/main/resources/ui/admin-web/test/unit/controllers/CreateViewInstanceCtrl.js
>  f8ccc24 
>   
> ambari-admin/src/main/resources/ui/admin-web/test/unit/controllers/mainCtrl_test.js
>  f03dcb5 
>   
> ambari-admin/src/main/resources/ui/admin-web/test/unit/services/Utility_test.js
>  6b0b17f 
>   ambari-agent/conf/unix/ambari-agent.ini 9a0b537 
>   ambari-agent/pom.xml a8ed7f1 
>   ambari-agent/src/main/python/ambari_agent/ActionQueue.py a9f1084 
>   ambari-agent/src/main/python/ambari_agent/AmbariConfig.py 6e4d74a 
>   ambari-agent/src/main/python/ambari_agent/Controller.py cece30c 
>   ambari-agent/src/main/python/ambari_agent/HostCleanup.py cca79a8 
>   ambari-agent/src/main/python/ambari_agent/PythonReflectiveExecutor.py 
> b27d7d1 
>   ambari-agent/src/main/python/ambari_agent/StatusCommandsExecutor.py 5d3607a 
>   ambari-agent/src/main/python/ambari_agent/apscheduler/scheduler.py f787a2d 
>   ambari-agent/src/main/python/ambari_agent/apscheduler/threadpool.py cb19888 
>   ambari-agent/src/main/python/ambari_agent/main.py 04e0c2c 
>   ambari-agent/src/main/python/ambari_agent/security.py 72ae954 
>   ambari-agent/src/test/python/ambari_agent/TestActionQueue.py 4a63f7c 
>   ambari-agent/src/test/python/ambari_agent/TestAlerts.py 2bddc43 
>   ambari-agent/src/test/python/ambari_agent/TestAmbariConfig.py c89644f 
>   ambari-agent/src/test/python/ambari_agent/TestController.py b47af03 
>   ambari-agent/src/test/python/ambari_agent/TestMain.py 6f38410 
>   ambari-agent/src/test/python/ambari_agent/TestRecoveryManager.py 7963a17 
>   ambari-agent/src/test/python/resource_management/TestDataStructureUtils.py 
> a4bdc1c 
>   
> ambari-agent/src/test/python/resource_management/TestPropertiesFileResource.py
>  286ff37 
>   ambari-client/groovy-client/pom.xml fa89a73 
>   ambari-client/pom.xml 3c6fd99 
>   ambari-client/python-client/pom.xml b8cfb94 
>   ambari-common/src/main/python/ambari_commons/ambari_metrics_helper.py 
> 2eb0b6d 
>   ambari-common/src/main/python/ambari_commons/parallel_processing.py c5a95de 
>   ambari-common/src/main/python/resource_management/core/exceptions.py 
> a59611d 
>   
>

Review Request 53592: Allow Ambari to manage Kafka's Custom JAAS Config

2016-11-08 Thread Sumit Mohanty


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53592/
---

Review request for Ambari, Sriharsha Chintalapani and Sid Wagle.


Bugs: AMBARI-18829
https://issues.apache.org/jira/browse/AMBARI-18829


Repository: ambari


Description
---

Allow Ambari to manage Kafka's Custom JAAS Config.
Move the j2 templates into config properties.

I intentionally left the j2 templates and logic to fall back to j2 template for 
now. They can be removed later.


Diffs
-

  LICENSE.txt f05016f 
  ambari-admin/pom.xml 4a8bba3 
  ambari-admin/src/main/resources/ui/admin-web/app/index.html b3f1bd1 
  ambari-admin/src/main/resources/ui/admin-web/app/scripts/app.js 80e2813 
  
ambari-admin/src/main/resources/ui/admin-web/app/scripts/controllers/ambariViews/CreateViewInstanceCtrl.js
 94b8cc1 
  
ambari-admin/src/main/resources/ui/admin-web/app/scripts/controllers/clusters/ExportBlueprintCtrl.js
 a1e4fa0 
  
ambari-admin/src/main/resources/ui/admin-web/app/scripts/controllers/mainCtrl.js
 c945644 
  
ambari-admin/src/main/resources/ui/admin-web/app/scripts/controllers/users/UsersShowCtrl.js
 f7ed7ee 
  ambari-admin/src/main/resources/ui/admin-web/app/scripts/i18n.config.js 
cd9b922 
  ambari-admin/src/main/resources/ui/admin-web/app/scripts/routes.js a1ca59c 
  ambari-admin/src/main/resources/ui/admin-web/app/scripts/services/Cluster.js 
02c231a 
  
ambari-admin/src/main/resources/ui/admin-web/app/scripts/services/PermissionLoader.js
 9cc04e4 
  ambari-admin/src/main/resources/ui/admin-web/app/scripts/services/Utility.js 
7ff10f8 
  
ambari-admin/src/main/resources/ui/admin-web/app/views/ambariViews/create.html 
25f1487 
  
ambari-admin/src/main/resources/ui/admin-web/app/views/ambariViews/listTable.html
 e03f035 
  
ambari-admin/src/main/resources/ui/admin-web/app/views/clusters/exportBlueprint.html
 6ccbffe 
  ambari-admin/src/main/resources/ui/admin-web/app/views/groups/edit.html 
78a6884 
  ambari-admin/src/main/resources/ui/admin-web/app/views/leftNavbar.html 
a7fc18a 
  ambari-admin/src/main/resources/ui/admin-web/app/views/users/create.html 
cc5d8d4 
  
ambari-admin/src/main/resources/ui/admin-web/test/unit/controllers/CreateViewInstanceCtrl.js
 f8ccc24 
  
ambari-admin/src/main/resources/ui/admin-web/test/unit/controllers/mainCtrl_test.js
 f03dcb5 
  
ambari-admin/src/main/resources/ui/admin-web/test/unit/services/Utility_test.js 
6b0b17f 
  ambari-agent/conf/unix/ambari-agent.ini 9a0b537 
  ambari-agent/pom.xml a8ed7f1 
  ambari-agent/src/main/python/ambari_agent/ActionQueue.py a9f1084 
  ambari-agent/src/main/python/ambari_agent/AmbariConfig.py 6e4d74a 
  ambari-agent/src/main/python/ambari_agent/Controller.py cece30c 
  ambari-agent/src/main/python/ambari_agent/HostCleanup.py cca79a8 
  ambari-agent/src/main/python/ambari_agent/PythonReflectiveExecutor.py b27d7d1 
  ambari-agent/src/main/python/ambari_agent/StatusCommandsExecutor.py 5d3607a 
  ambari-agent/src/main/python/ambari_agent/apscheduler/scheduler.py f787a2d 
  ambari-agent/src/main/python/ambari_agent/apscheduler/threadpool.py cb19888 
  ambari-agent/src/main/python/ambari_agent/main.py 04e0c2c 
  ambari-agent/src/main/python/ambari_agent/security.py 72ae954 
  ambari-agent/src/test/python/ambari_agent/TestActionQueue.py 4a63f7c 
  ambari-agent/src/test/python/ambari_agent/TestAlerts.py 2bddc43 
  ambari-agent/src/test/python/ambari_agent/TestAmbariConfig.py c89644f 
  ambari-agent/src/test/python/ambari_agent/TestController.py b47af03 
  ambari-agent/src/test/python/ambari_agent/TestMain.py 6f38410 
  ambari-agent/src/test/python/ambari_agent/TestRecoveryManager.py 7963a17 
  ambari-agent/src/test/python/resource_management/TestDataStructureUtils.py 
a4bdc1c 
  
ambari-agent/src/test/python/resource_management/TestPropertiesFileResource.py 
286ff37 
  ambari-client/groovy-client/pom.xml fa89a73 
  ambari-client/pom.xml 3c6fd99 
  ambari-client/python-client/pom.xml b8cfb94 
  ambari-common/src/main/python/ambari_commons/ambari_metrics_helper.py 2eb0b6d 
  ambari-common/src/main/python/ambari_commons/parallel_processing.py c5a95de 
  ambari-common/src/main/python/resource_management/core/exceptions.py a59611d 
  
ambari-common/src/main/python/resource_management/libraries/functions/curl_krb_request.py
 557db58 
  
ambari-common/src/main/python/resource_management/libraries/functions/data_structure_utils.py
 59e01aa 
  
ambari-common/src/main/python/resource_management/libraries/functions/flume_agent_helper.py
 80e7b28 
  
ambari-common/src/main/python/resource_management/libraries/functions/get_user_call_output.py
 7b7acae 
  
ambari-common/src/main/python/resource_management/libraries/providers/properties_file.py
 2bf1f5b 
  
ambari-common/src/main/python/resource_management/libraries/resources/properties_file.py
 826fcf8

Re: Review Request 53565: Add ability to Skip Host check option on bootstrap page

2016-11-08 Thread Xi Wang


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53565/#review155386
---


Ship it!




Ship It!

- Xi Wang


On Nov. 8, 2016, 12:09 p.m., Jaimin Jetly wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53565/
> ---
> 
> (Updated Nov. 8, 2016, 12:09 p.m.)
> 
> 
> Review request for Ambari, Richard Zang, Vivek Ratnavel Subramanian, Xi Wang, 
> and Yusaku Sako.
> 
> 
> Bugs: AMBARI-18817
> https://issues.apache.org/jira/browse/AMBARI-18817
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> As part of this work experimental flag has been added by which host checks 
> can be disabled while going thorugh "Add Host Wizard"
> 
> 
> Diffs
> -
> 
>   ambari-web/app/config.js 06faf27 
>   ambari-web/app/controllers/wizard/step3_controller.js 0ccf2ed 
>   ambari-web/app/templates/wizard/step3.hbs f5648e1 
> 
> Diff: https://reviews.apache.org/r/53565/diff/
> 
> 
> Testing
> ---
> 
> manually verified the patch on the lcuster
> verified that all ambar-web unit tests passes
>   29248 tests complete (30 seconds)
>   154 tests pending
> 
> 
> Thanks,
> 
> Jaimin Jetly
> 
>

Re: Review Request 51815: Introduce conditional dependencies in stack defition to handle blueprint validation gracefully



> On Oct. 13, 2016, 1:24 p.m., Nate Cole wrote:
> > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml,
> >  lines 43-47
> > 
> >
> > This is a bit of an odd syntax.  the  element should be a 
> > different class then you don't need the  element.  Now, 
> > these metainfo.xml files don't have schema for validation, but java XML is 
> > able to use correct classes based on the element name.  So for example, you 
> > would have two types such that:
> > 
> > 
> > 
> >   
> > 
> >   
> >   
> > 
> >   
> > 
> > 

Hello Nate,
Could you please provide your feedback on the updated patch?

Thank you,
Amruta


- Amruta


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51815/#review152489
---


On Nov. 1, 2016, 6:57 p.m., Amruta Borkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51815/
> ---
> 
> (Updated Nov. 1, 2016, 6:57 p.m.)
> 
> 
> Review request for Ambari, Shantanu Mundkur, Di Li, Jayush Luniya, Juanjo  
> Marron, Laszlo Puskas, Nate Cole, and Robert Nettleton.
> 
> 
> Bugs: AMBARI-18355
> https://issues.apache.org/jira/browse/AMBARI-18355
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Currently stack definitions do not list conditional dependencies, adding 
> those to the stack definitions would make it easy to validate errors in case 
> of blueprint deployment. Please refer to document attached to Jira
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/DependencyConditionInfo.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/DependencyInfo.java
>  e3db662 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintValidatorImpl.java
>  a5f33ff 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml 
> 9c1387d 
>   
> ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintImplTest.java
>  ff9af17 
>   
> ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintValidatorImplTest.java
>  b1de8ef 
> 
> Diff: https://reviews.apache.org/r/51815/diff/
> 
> 
> Testing
> ---
> 
> Written Junit test cases. Perfomred manual testing to check the Namenode HA 
> component dependency. Was able to proceed with the installation for valid 
> blueprint. and got validation error message while registering blueprint when 
> the Blueprint did not satisfy the conditional dependencies.
> 
> 
> Thanks,
> 
> Amruta Borkar
> 
>

Re: Review Request 53213: Exporting blueprint from kerberos enabled cluster, exports hardcoded values cluster name and realm in principal_name property


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53213/
---

(Updated Nov. 9, 2016, 12:11 a.m.)


Review request for Ambari, Di Li, Jaimin Jetly, Jayush Luniya, Robert Levas, 
and Robert Nettleton.


Bugs: AMBARI-18692
https://issues.apache.org/jira/browse/AMBARI-18692


Repository: ambari


Description
---

Exporting blueprint from kerberos enabled cluster, exports hardcoded values 
cluster name and realm in principal_name property.
When the same blueprint is used to create another cluster with different name, 
service start fail with following error:
"resource_management.core.exceptions.Fail: Execution of '/usr/bin/kinit -kt 
/etc/security/keytabs/hdfs.headless.keytab [keytab_name_in_blueprint] eturned 
1. kinit: Keytab contains no suitable keys for [keytab_name_in_blueprint] while 
getting initial credentials"


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
 f890326 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/Stack.java
 16f75ee 
  ambari-server/src/main/java/org/apache/ambari/server/state/PropertyInfo.java 
81de76c 
  
ambari-server/src/main/resources/common-services/ACCUMULO/1.6.1.2.2.0/configuration/accumulo-env.xml
 1d330dd 
  
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/configuration/infra-solr-env.xml
 d7ae236 
  
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-hbase-security-site.xml
 2be101b 
  
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/configuration/application-properties.xml
 9efa4f9 
  
ambari-server/src/main/resources/common-services/DRUID/0.9.2/configuration/druid-common.xml
 8712c7d 
  
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/configuration/hbase-env.xml
 24bd563 
  
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/configuration/hadoop-env.xml
 afaaee8 
  
ambari-server/src/main/resources/common-services/KAFKA/0.8.1/configuration/kafka-env.xml
 e8a76b6 
  
ambari-server/src/main/resources/common-services/KERBEROS/1.10.3-10/configuration/kerberos-env.xml
 7016437 
  
ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/configuration/knox-env.xml
 2e5a026 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logfeeder-env.xml
 ee885e3 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-env.xml
 c5b9b4e 
  
ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-admin-site.xml
 36ebc8c 
  
ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-tagsync-site.xml
 a0bd322 
  
ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-ugsync-site.xml
 d267b75 
  
ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/configuration/kms-site.xml
 1bce4e6 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/configuration/spark-defaults.xml
 b85cf50 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/configuration/spark-env.xml
 71a42f7 
  
ambari-server/src/main/resources/common-services/SPARK2/2.0.0/configuration/spark2-defaults.xml
 a5115af 
  
ambari-server/src/main/resources/common-services/SPARK2/2.0.0/configuration/spark2-env.xml
 b78fd79 
  
ambari-server/src/main/resources/common-services/STORM/0.9.1/configuration/storm-env.xml
 5d0b2ff 
  
ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/configuration/zeppelin-env.xml
 c03d2dc 
  
ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5/configuration/zookeeper-env.xml
 696b28a 
  ambari-server/src/main/resources/configuration-schema.xsd daba29f 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
 5bedb9d 

Diff: https://reviews.apache.org/r/53213/diff/


Testing
---

Tested manually. Suitable keytabs are generated automatically when not 
mentioned in blueprint. Service starts succeeded with a blueprint exported with 
the code change. 
Modified existing unit test cases.


Thanks,

Amruta Borkar

Re: Review Request 53213: Exporting blueprint from kerberos enabled cluster, exports hardcoded values cluster name and realm in principal_name property



> On Oct. 31, 2016, 3:31 p.m., Robert Nettleton wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java,
> >  line 2993
> > 
> >
> > Are we sure that all properties that include the string "principal" 
> > should be excluded?  
> > 
> > Are all of these properties certain to have the cluster-specific name 
> > that would be an issue for portability?  
> > 
> > Given the general nature of this property name matcher, I'd be a little 
> > concerned that maybe more is being filtered out than necessary.
> > 
> > I'm not sure that this is definitely a problem, but it might be a good 
> > idea to double-check the stack definitions, to make sure that only the 
> > problematic properties are being excluded.

Changed the patch according to Robert's suggestions, which would avoid this 
issue.


- Amruta


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53213/#review154268
---


On Oct. 28, 2016, 5:50 p.m., Amruta Borkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53213/
> ---
> 
> (Updated Oct. 28, 2016, 5:50 p.m.)
> 
> 
> Review request for Ambari, Di Li, Robert Levas, and Robert Nettleton.
> 
> 
> Bugs: AMBARI-18692
> https://issues.apache.org/jira/browse/AMBARI-18692
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Exporting blueprint from kerberos enabled cluster, exports hardcoded values 
> cluster name and realm in principal_name property.
> When the same blueprint is used to create another cluster with different 
> name, service start fail with following error:
> "resource_management.core.exceptions.Fail: Execution of '/usr/bin/kinit -kt 
> /etc/security/keytabs/hdfs.headless.keytab [keytab_name_in_blueprint] eturned 
> 1. kinit: Keytab contains no suitable keys for [keytab_name_in_blueprint] 
> while getting initial credentials"
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
>  f890326 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
>  5bedb9d 
> 
> Diff: https://reviews.apache.org/r/53213/diff/
> 
> 
> Testing
> ---
> 
> Tested manually. Suitable keytabs are generated automatically when not 
> mentioned in blueprint. Service starts succeeded with a blueprint exported 
> with the code change. 
> Modified existing unit test cases.
> 
> 
> Thanks,
> 
> Amruta Borkar
> 
>

Re: Review Request 53213: Exporting blueprint from kerberos enabled cluster, exports hardcoded values cluster name and realm in principal_name property



> On Oct. 31, 2016, 4:27 p.m., Robert Levas wrote:
> > This seems dangerous.  
> > 
> > I think it would be a better option to make this explicit by using a 
> > configuration attribute. For example:
> > 
> > '''
> > 
> >   ...
> >   kerberos_principal
> >   ...
> > 
> > '''
> > 
> > However I am not sure if this data is available at the time you would need 
> > it.
> 
> Amruta Borkar wrote:
> Hello Robert,
> Would it be ok if a new property type is defined to identify kerberos 
> principal EX: kerberos-principal rather than 
> defining it in  ? As we currently use 
> password to identify and filter out password 
> references while blueprint export.
> 
> Robert Levas wrote:
> I think that will work too but we might need some more expertice on this. 
>  After making the change, can you add Jayush Luniya and Jaimin Jetly to the 
> review? Also, make sure you update `configuration-schema.xsd` and 
> `org.apache.ambari.server.state.PropertyInfo.PropertyType`

Updated the the patch based on suggestions. Also adding Jayush Luniya and 
Jaimin Jetly.


- Amruta


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53213/#review154274
---


On Oct. 28, 2016, 5:50 p.m., Amruta Borkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53213/
> ---
> 
> (Updated Oct. 28, 2016, 5:50 p.m.)
> 
> 
> Review request for Ambari, Di Li, Robert Levas, and Robert Nettleton.
> 
> 
> Bugs: AMBARI-18692
> https://issues.apache.org/jira/browse/AMBARI-18692
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Exporting blueprint from kerberos enabled cluster, exports hardcoded values 
> cluster name and realm in principal_name property.
> When the same blueprint is used to create another cluster with different 
> name, service start fail with following error:
> "resource_management.core.exceptions.Fail: Execution of '/usr/bin/kinit -kt 
> /etc/security/keytabs/hdfs.headless.keytab [keytab_name_in_blueprint] eturned 
> 1. kinit: Keytab contains no suitable keys for [keytab_name_in_blueprint] 
> while getting initial credentials"
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
>  f890326 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
>  5bedb9d 
> 
> Diff: https://reviews.apache.org/r/53213/diff/
> 
> 
> Testing
> ---
> 
> Tested manually. Suitable keytabs are generated automatically when not 
> mentioned in blueprint. Service starts succeeded with a blueprint exported 
> with the code change. 
> Modified existing unit test cases.
> 
> 
> Thanks,
> 
> Amruta Borkar
> 
>

Re: Review Request 53584: Ensure HAWQ -PXF service check uses a tmp directory

2016-11-08 Thread Lav Jain


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53584/#review155371
---


Ship it!




Ship It!

- Lav Jain


On Nov. 8, 2016, 8:26 p.m., Matt wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53584/
> ---
> 
> (Updated Nov. 8, 2016, 8:26 p.m.)
> 
> 
> Review request for Ambari and Lav Jain.
> 
> 
> Bugs: AMBARI-18828
> https://issues.apache.org/jira/browse/AMBARI-18828
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Ensure HAWQ -PXF service check uses a tmp directory and that this directory 
> is deleted after the service check has been completed.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/hawq_constants.py
>  3ced8d1 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/service_check.py
>  6e7ea0f 
> 
> Diff: https://reviews.apache.org/r/53584/diff/
> 
> 
> Testing
> ---
> 
> Manually tested
> 
> 
> Thanks,
> 
> Matt
> 
>

Re: Review Request 53588: Allow acceptor / seclector configuration for API and agent connectors

2016-11-08 Thread Sid Wagle


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53588/#review155370
---



Need to change the code, seems like the agent always ping non-secure port to 
download cert, so need to configure both connectors.

- Sid Wagle


On Nov. 8, 2016, 10:45 p.m., Sid Wagle wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53588/
> ---
> 
> (Updated Nov. 8, 2016, 10:45 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Myroslav 
> Papirkovskyy, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-18827
> https://issues.apache.org/jira/browse/AMBARI-18827
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> _Objectives_:
> - Allow acceptors for agent and api connectors to be configurable
> - The thread pool configuration did not take into account both 2-way and 
> 1-way connectors are configured for agent every time although only one is 
> used and not a mixed-mode. This causes insufficient threads in agent 
> threadpool for a high cpu core environment.
> - Includes refactoring to remove code duplication.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
>  8857e19 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  ce3fe85 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/unsecured/rest/ConnectionInfo.java
>  a1b5a6e 
>   
> ambari-server/src/test/java/org/apache/ambari/server/configuration/ConfigurationTest.java
>  844f022 
> 
> Diff: https://reviews.apache.org/r/53588/diff/
> 
> 
> Testing
> ---
> 
> Verified manually.
> 
> 2-core VM ==>
> Before patch:
> [root@swagle-test-1 ~]# cat ~/jstack_before.out | grep 
> "ambari-agent.*Acceptor" | wc -l
> 4
> [root@swagle-test-1 ~]# cat ~/jstack_before.out | grep 
> "ambari-agent.*Selector" | wc -l
> 4
> [root@swagle-test-1 ambari-server]# cat ~/jstack_before.out | grep 
> "ambari-client.*Selector" | wc -l
> 1
> [root@swagle-test-1 ambari-server]# cat ~/jstack_before.out | grep 
> "ambari-client.*Acceptor" | wc -l
> 1
> 
> Default after patch:
> [root@swagle-test-1 ambari-server]# cat ~/jstack_after.out | grep 
> "ambari-client.*Acceptor" | wc -l
> 1
> [root@swagle-test-1 ambari-server]# cat ~/jstack_after.out | grep 
> "ambari-agent.*Acceptor" | wc -l
> 1
> [root@swagle-test-1 ambari-server]# cat ~/jstack_after.out | grep 
> "ambari-client.*Selector" | wc -l
> 1
> [root@swagle-test-1 ambari-server]# cat ~/jstack_after.out | grep 
> "ambari-agent.*Selector" | wc -l
> 1
> 
> Setting configurations = 2:
> [root@swagle-test-1 ambari-server]# cat ~/jstack_configured.out | grep 
> "ambari-agent.*Selector" | wc -l
> 2
> [root@swagle-test-1 ambari-server]# cat ~/jstack_configured.out | grep 
> "ambari-client.*Selector" | wc -l
> 2
> [root@swagle-test-1 ambari-server]# cat ~/jstack_configured.out | grep 
> "ambari-agent.*Acceptor" | wc -l
> 2
> [root@swagle-test-1 ambari-server]# cat ~/jstack_configured.out | grep 
> "ambari-client.*Acceptor" | wc -l
> 2
> 
> 
> Thanks,
> 
> Sid Wagle
> 
>

Re: Review Request 53588: Allow acceptor / seclector configuration for API and agent connectors

2016-11-08 Thread Sid Wagle


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53588/
---

(Updated Nov. 8, 2016, 10:45 p.m.)


Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Myroslav 
Papirkovskyy, and Sumit Mohanty.


Bugs: AMBARI-18827
https://issues.apache.org/jira/browse/AMBARI-18827


Repository: ambari


Description (updated)
---

_Objectives_:
- Allow acceptors for agent and api connectors to be configurable
- The thread pool configuration did not take into account both 2-way and 1-way 
connectors are configured for agent every time although only one is used and 
not a mixed-mode. This causes insufficient threads in agent threadpool for a 
high cpu core environment.
- Includes refactoring to remove code duplication.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
 8857e19 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
 ce3fe85 
  
ambari-server/src/main/java/org/apache/ambari/server/security/unsecured/rest/ConnectionInfo.java
 a1b5a6e 
  
ambari-server/src/test/java/org/apache/ambari/server/configuration/ConfigurationTest.java
 844f022 

Diff: https://reviews.apache.org/r/53588/diff/


Testing
---

Verified manually.

2-core VM ==>
Before patch:
[root@swagle-test-1 ~]# cat ~/jstack_before.out | grep "ambari-agent.*Acceptor" 
| wc -l
4
[root@swagle-test-1 ~]# cat ~/jstack_before.out | grep "ambari-agent.*Selector" 
| wc -l
4
[root@swagle-test-1 ambari-server]# cat ~/jstack_before.out | grep 
"ambari-client.*Selector" | wc -l
1
[root@swagle-test-1 ambari-server]# cat ~/jstack_before.out | grep 
"ambari-client.*Acceptor" | wc -l
1

Default after patch:
[root@swagle-test-1 ambari-server]# cat ~/jstack_after.out | grep 
"ambari-client.*Acceptor" | wc -l
1
[root@swagle-test-1 ambari-server]# cat ~/jstack_after.out | grep 
"ambari-agent.*Acceptor" | wc -l
1
[root@swagle-test-1 ambari-server]# cat ~/jstack_after.out | grep 
"ambari-client.*Selector" | wc -l
1
[root@swagle-test-1 ambari-server]# cat ~/jstack_after.out | grep 
"ambari-agent.*Selector" | wc -l
1

Setting configurations = 2:
[root@swagle-test-1 ambari-server]# cat ~/jstack_configured.out | grep 
"ambari-agent.*Selector" | wc -l
2
[root@swagle-test-1 ambari-server]# cat ~/jstack_configured.out | grep 
"ambari-client.*Selector" | wc -l
2
[root@swagle-test-1 ambari-server]# cat ~/jstack_configured.out | grep 
"ambari-agent.*Acceptor" | wc -l
2
[root@swagle-test-1 ambari-server]# cat ~/jstack_configured.out | grep 
"ambari-client.*Acceptor" | wc -l
2


Thanks,

Sid Wagle

Review Request 53588: Allow acceptor / seclector configuration for API and agent connectors

2016-11-08 Thread Sid Wagle


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53588/
---

Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Myroslav 
Papirkovskyy, and Sumit Mohanty.


Bugs: AMBARI-18827
https://issues.apache.org/jira/browse/AMBARI-18827


Repository: ambari


Description
---

_Objectives_:
- Allow acceptors for agent and api connectors to be configurable
- The thread pool configuration did not take into account both 2-way and 1-way 
connectors are configured for agent every time although only one is used and 
not a mixed-mode. This causes insufficient threads in agent threadpool for a 
high cpu core environment.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
 8857e19 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
 ce3fe85 
  
ambari-server/src/main/java/org/apache/ambari/server/security/unsecured/rest/ConnectionInfo.java
 a1b5a6e 
  
ambari-server/src/test/java/org/apache/ambari/server/configuration/ConfigurationTest.java
 844f022 

Diff: https://reviews.apache.org/r/53588/diff/


Testing
---

Verified manually.

2-core VM ==>
Before patch:
[root@swagle-test-1 ~]# cat ~/jstack_before.out | grep "ambari-agent.*Acceptor" 
| wc -l
4
[root@swagle-test-1 ~]# cat ~/jstack_before.out | grep "ambari-agent.*Selector" 
| wc -l
4
[root@swagle-test-1 ambari-server]# cat ~/jstack_before.out | grep 
"ambari-client.*Selector" | wc -l
1
[root@swagle-test-1 ambari-server]# cat ~/jstack_before.out | grep 
"ambari-client.*Acceptor" | wc -l
1

Default after patch:
[root@swagle-test-1 ambari-server]# cat ~/jstack_after.out | grep 
"ambari-client.*Acceptor" | wc -l
1
[root@swagle-test-1 ambari-server]# cat ~/jstack_after.out | grep 
"ambari-agent.*Acceptor" | wc -l
1
[root@swagle-test-1 ambari-server]# cat ~/jstack_after.out | grep 
"ambari-client.*Selector" | wc -l
1
[root@swagle-test-1 ambari-server]# cat ~/jstack_after.out | grep 
"ambari-agent.*Selector" | wc -l
1

Setting configurations = 2:
[root@swagle-test-1 ambari-server]# cat ~/jstack_configured.out | grep 
"ambari-agent.*Selector" | wc -l
2
[root@swagle-test-1 ambari-server]# cat ~/jstack_configured.out | grep 
"ambari-client.*Selector" | wc -l
2
[root@swagle-test-1 ambari-server]# cat ~/jstack_configured.out | grep 
"ambari-agent.*Acceptor" | wc -l
2
[root@swagle-test-1 ambari-server]# cat ~/jstack_configured.out | grep 
"ambari-client.*Acceptor" | wc -l
2


Thanks,

Sid Wagle

Re: Review Request 53564: Ranger install asks to populate AD domain and marks its required when no AD is in play

2016-11-08 Thread Jayush Luniya


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53564/#review155367
---


Ship it!




Ship It!

- Jayush Luniya


On Nov. 8, 2016, 2:45 a.m., Jaimin Jetly wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53564/
> ---
> 
> (Updated Nov. 8, 2016, 2:45 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya, Richard Zang, Vivek Ratnavel 
> Subramanian, Xi Wang, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-18816
> https://issues.apache.org/jira/browse/AMBARI-18816
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Ranger install asks to populate AD domain and marks its required when no AD 
> is in play
> 
> 
> Diffs
> -
> 
>   ambari-web/app/data/custom_stack_map.js 33b642c 
>   ambari-web/app/utils/config.js e83d42c 
>   ambari-web/test/utils/config_test.js 35d272d 
> 
> Diff: https://reviews.apache.org/r/53564/diff/
> 
> 
> Testing
> ---
> 
> Added new unit test.
> Verified that all existing unit tests passes:
> 
> 
>   29250 tests complete (29 seconds)
>   154 tests pending
> 
> 
> Thanks,
> 
> Jaimin Jetly
> 
>

Re: Review Request 53583: Optionally force username from LDAP authentication data to be lowercase in Ambari



> On Nov. 8, 2016, 3:51 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AmbariLdapBindAuthenticator.java,
> >  line 80
> > 
> >
> > Out of curiousity, can LDAP contain non-latin characters which might 
> > not convert properly to lowercase?
> 
> Robert Levas wrote:
> I suppose non-latin characters are a possibly, but I don't think we 
> typically see that.  Also, if this was an issue we would have bigger problems 
> since the default charater set for MySQL is LATIN1.
> 
> ```
> mysql> SELECT  T.table_schema, T.table_name, CCSA.character_set_name FROM 
> information_schema.`TABLES` T,
> ->information_schema.`COLLATION_CHARACTER_SET_APPLICABILITY` 
> CCSA
> -> WHERE CCSA.collation_name = T.table_collation
> ->   AND T.table_schema = "ambari"
> ->   AND T.table_name = "users";
> +--+++
> | table_schema | table_name | character_set_name |
> +--+++
> | ambari   | users  | latin1 |
> +--+++
> 1 row in set (0.00 sec)
> ```

Actually, many MySQL installations (especially those 5.6 and greater) are done 
with UTF-8 as a charset. It's just something I always think about when using 
toLowerCase() ...


- Jonathan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53583/#review155354
---


On Nov. 8, 2016, 2:09 p.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53583/
> ---
> 
> (Updated Nov. 8, 2016, 2:09 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Laszlo Puskas, Nate Cole, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-18813
> https://issues.apache.org/jira/browse/AMBARI-18813
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Optionally force username from LDAP authentication data to be lowercase in 
> Ambari based on LDAP import configuration.
> 
> In some cases the username declared in the relevant LDAP object is in all 
> uppercase characters when the local Hadoop cluster expects the username to be 
> all lowercase. As of Ambari 2.4.0, the username specified from the LDAP data 
> is used to override the username known to Ambari.  This overwritten data may 
> be in all uppercase characters, potentially breaking local username 
> conventions. 
> 
> To help this scenario, provide a configuration option to force the username 
> obtained from the LDAP object to be converted to all lowercase character. 
> 
> For example `authentication.ldap.username.forceLowercase`.
> 
> This optional configuration value is to default to false to maintain current 
> functionality.
> 
> 
> Diffs
> -
> 
>   ambari-server/docs/configuration/index.md 52b4744 
>   
> ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
>  3103b6d 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AmbariLdapBindAuthenticator.java
>  917471b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/LdapServerProperties.java
>  a3086693 
>   
> ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariLdapBindAuthenticatorTest.java
>  6e4b836 
> 
> Diff: https://reviews.apache.org/r/53583/diff/
> 
> 
> Testing
> ---
> 
> Manually tested...
> 
> # Local test results:
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 37:51.683s
> [INFO] Finished at: Tue Nov 08 10:51:17 EST 2016
> [INFO] Final Memory: 62M/834M
> [INFO] 
> 
> 
> # Jenkins test result: PENDING
> 
> 
> Thanks,
> 
> Robert Levas
> 
>

Re: Review Request 53583: Optionally force username from LDAP authentication data to be lowercase in Ambari


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53583/#review155365
---


Ship it!




Ship It!

- Jonathan Hurley


On Nov. 8, 2016, 2:09 p.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53583/
> ---
> 
> (Updated Nov. 8, 2016, 2:09 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Laszlo Puskas, Nate Cole, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-18813
> https://issues.apache.org/jira/browse/AMBARI-18813
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Optionally force username from LDAP authentication data to be lowercase in 
> Ambari based on LDAP import configuration.
> 
> In some cases the username declared in the relevant LDAP object is in all 
> uppercase characters when the local Hadoop cluster expects the username to be 
> all lowercase. As of Ambari 2.4.0, the username specified from the LDAP data 
> is used to override the username known to Ambari.  This overwritten data may 
> be in all uppercase characters, potentially breaking local username 
> conventions. 
> 
> To help this scenario, provide a configuration option to force the username 
> obtained from the LDAP object to be converted to all lowercase character. 
> 
> For example `authentication.ldap.username.forceLowercase`.
> 
> This optional configuration value is to default to false to maintain current 
> functionality.
> 
> 
> Diffs
> -
> 
>   ambari-server/docs/configuration/index.md 52b4744 
>   
> ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
>  3103b6d 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AmbariLdapBindAuthenticator.java
>  917471b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/LdapServerProperties.java
>  a3086693 
>   
> ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariLdapBindAuthenticatorTest.java
>  6e4b836 
> 
> Diff: https://reviews.apache.org/r/53583/diff/
> 
> 
> Testing
> ---
> 
> Manually tested...
> 
> # Local test results:
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 37:51.683s
> [INFO] Finished at: Tue Nov 08 10:51:17 EST 2016
> [INFO] Final Memory: 62M/834M
> [INFO] 
> 
> 
> # Jenkins test result: PENDING
> 
> 
> Thanks,
> 
> Robert Levas
> 
>

Re: Review Request 53502: AMBARI-18681 Add host structured_out to manual task


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53502/#review155362
---


Ship it!




This has been pushed to branch-feature-AMBARI-18634, trunk, and branch-2.5.  
Please close this review and associated jira.

- Nate Cole


On Nov. 7, 2016, 10:42 p.m., Duc Le wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53502/
> ---
> 
> (Updated Nov. 7, 2016, 10:42 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, and Nate 
> Cole.
> 
> 
> Bugs: AMBARI-18681
> https://issues.apache.org/jira/browse/AMBARI-18681
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> AMBARI-18681 Add host structured_out to manual task
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/HostOrderGrouping.java
>  b999f2f 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/UpgradeHelperTest.java
>  78969e4 
> 
> Diff: https://reviews.apache.org/r/53502/diff/
> 
> 
> Testing
> ---
> 
> Unit tests passing.
> 
> ...
> Tests run: 28, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 209.464 sec 
> - in org.apache.ambari.server.state.UpgradeHelperTest
> ...
> Tests run: 4727, Failures: 0, Errors: 0, Skipped: 42
> 
> 
> Thanks,
> 
> Duc Le
> 
>

Re: Review Request 53583: Optionally force username from LDAP authentication data to be lowercase in Ambari


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53583/#review155361
---


Ship it!




Ship It!

- Nate Cole


On Nov. 8, 2016, 2:09 p.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53583/
> ---
> 
> (Updated Nov. 8, 2016, 2:09 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Laszlo Puskas, Nate Cole, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-18813
> https://issues.apache.org/jira/browse/AMBARI-18813
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Optionally force username from LDAP authentication data to be lowercase in 
> Ambari based on LDAP import configuration.
> 
> In some cases the username declared in the relevant LDAP object is in all 
> uppercase characters when the local Hadoop cluster expects the username to be 
> all lowercase. As of Ambari 2.4.0, the username specified from the LDAP data 
> is used to override the username known to Ambari.  This overwritten data may 
> be in all uppercase characters, potentially breaking local username 
> conventions. 
> 
> To help this scenario, provide a configuration option to force the username 
> obtained from the LDAP object to be converted to all lowercase character. 
> 
> For example `authentication.ldap.username.forceLowercase`.
> 
> This optional configuration value is to default to false to maintain current 
> functionality.
> 
> 
> Diffs
> -
> 
>   ambari-server/docs/configuration/index.md 52b4744 
>   
> ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
>  3103b6d 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AmbariLdapBindAuthenticator.java
>  917471b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/LdapServerProperties.java
>  a3086693 
>   
> ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariLdapBindAuthenticatorTest.java
>  6e4b836 
> 
> Diff: https://reviews.apache.org/r/53583/diff/
> 
> 
> Testing
> ---
> 
> Manually tested...
> 
> # Local test results:
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 37:51.683s
> [INFO] Finished at: Tue Nov 08 10:51:17 EST 2016
> [INFO] Final Memory: 62M/834M
> [INFO] 
> 
> 
> # Jenkins test result: PENDING
> 
> 
> Thanks,
> 
> Robert Levas
> 
>

Re: Review Request 53578: MOTD Can Prevent WebHCat From Shutting Down


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53578/#review155359
---


Ship it!




Ship It!

- Nate Cole


On Nov. 8, 2016, 1:04 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53578/
> ---
> 
> (Updated Nov. 8, 2016, 1:04 p.m.)
> 
> 
> Review request for Ambari, Nate Cole and Robert Levas.
> 
> 
> Bugs: AMBARI-18824
> https://issues.apache.org/jira/browse/AMBARI-18824
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> There was code added in AMBARI-12695 (later corrected by AMBARI-18684) which 
> tries to kill WebHCat if it doesn't stop with its own stop command.
> 
> Both patches use the {{as_user}} function to wrap the call in a bash login:
> ```
> ps -p `su hcat -l -s /bin/bash -c 'cat /var/run/webhcat/webhcat.pid'` 
> >/dev/null 2>&1
> ```
> 
> This, however, causes problems where there is a MOTD:
> ```
> ERROR: Process ID list syntax error.
> * simple selection *  * selection by list *
> -A all processes  -C by command name
> -N negate selection   -G by real group ID (supports names)
> -a all w/ tty except session leaders  -U by real user ID (supports names)
> 
> ...
> 
> ```
> 
> That's not good - why is it giving us back a bad command syntax? Here's the 
> reason:
> ```
> [root@ijzs-c229todgsecha-r6-re1-3 ~]# su hcat -l -s /bin/bash -c 'cat 
> /var/run/webhcat/webhcat.pid'
> This is MOTD message :)
> 25672
> ```
> 
> *The MOTD is messing up the output from commands for the {{hcat}} user. 
> Because we rely on the error code of {{ps}}, the same non-zero code is used 
> for a bad command and a missing process ID*.
> 
> Since the PID is world-readable, we don't need to run the {{only_if}} checks 
> as a separate user.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat_service.py
>  fc6b933 
>   ambari-server/src/test/python/stacks/2.0.6/HIVE/test_webhcat_server.py 
> 0880a19 
> 
> Diff: https://reviews.apache.org/r/53578/diff/
> 
> 
> Testing
> ---
> 
> Tested with non-root agents and a MOTD
> 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 49.346 s
> [INFO] Finished at: 2016-11-08T12:49:12-05:00
> [INFO] Final Memory: 20M/439M
> [INFO] 
> 
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Re: Review Request 53583: Optionally force username from LDAP authentication data to be lowercase in Ambari



> On Nov. 8, 2016, 3:51 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java,
> >  line 3745
> > 
> >
> > Would a Boolean.parseBoolean(...) be easier to read here?

Probably.. and I suppose there is no threat of a `null`.  I'll fix.


> On Nov. 8, 2016, 3:51 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AmbariLdapBindAuthenticator.java,
> >  line 80
> > 
> >
> > Out of curiousity, can LDAP contain non-latin characters which might 
> > not convert properly to lowercase?

I suppose non-latin characters are a possibly, but I don't think we typically 
see that.  Also, if this was an issue we would have bigger problems since the 
default charater set for MySQL is LATIN1.

```
mysql> SELECT  T.table_schema, T.table_name, CCSA.character_set_name FROM 
information_schema.`TABLES` T,
->information_schema.`COLLATION_CHARACTER_SET_APPLICABILITY` CCSA
-> WHERE CCSA.collation_name = T.table_collation
->   AND T.table_schema = "ambari"
->   AND T.table_name = "users";
+--+++
| table_schema | table_name | character_set_name |
+--+++
| ambari   | users  | latin1 |
+--+++
1 row in set (0.00 sec)
```


- Robert


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53583/#review155354
---


On Nov. 8, 2016, 2:09 p.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53583/
> ---
> 
> (Updated Nov. 8, 2016, 2:09 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Laszlo Puskas, Nate Cole, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-18813
> https://issues.apache.org/jira/browse/AMBARI-18813
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Optionally force username from LDAP authentication data to be lowercase in 
> Ambari based on LDAP import configuration.
> 
> In some cases the username declared in the relevant LDAP object is in all 
> uppercase characters when the local Hadoop cluster expects the username to be 
> all lowercase. As of Ambari 2.4.0, the username specified from the LDAP data 
> is used to override the username known to Ambari.  This overwritten data may 
> be in all uppercase characters, potentially breaking local username 
> conventions. 
> 
> To help this scenario, provide a configuration option to force the username 
> obtained from the LDAP object to be converted to all lowercase character. 
> 
> For example `authentication.ldap.username.forceLowercase`.
> 
> This optional configuration value is to default to false to maintain current 
> functionality.
> 
> 
> Diffs
> -
> 
>   ambari-server/docs/configuration/index.md 52b4744 
>   
> ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
>  3103b6d 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AmbariLdapBindAuthenticator.java
>  917471b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/LdapServerProperties.java
>  a3086693 
>   
> ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariLdapBindAuthenticatorTest.java
>  6e4b836 
> 
> Diff: https://reviews.apache.org/r/53583/diff/
> 
> 
> Testing
> ---
> 
> Manually tested...
> 
> # Local test results:
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 37:51.683s
> [INFO] Finished at: Tue Nov 08 10:51:17 EST 2016
> [INFO] Final Memory: 62M/834M
> [INFO] 
> 
> 
> # Jenkins test result: PENDING
> 
> 
> Thanks,
> 
> Robert Levas
> 
>

Re: Review Request 53578: MOTD Can Prevent WebHCat From Shutting Down



> On Nov. 8, 2016, 1:33 p.m., Robert Levas wrote:
> > ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat_service.py,
> >  lines 86-88
> > 
> >
> > If removing `as_user`, does this need to indicate the user to run as?
> 
> Jonathan Hurley wrote:
> So, the `as_user` was only for the `only_if`. I'm not changing the 
> `Execute` itself since that's not the bug here. Since PID is accessible to 
> the world, then we don't need to run as a user to execute the 
> `process_id_exists_command`
> 
> Jonathan Hurley wrote:
> Since this is a hard kill command, running it using ambari-sudo.sh is 
> correct:
> ```'ambari-sudo.sh kill -9 `cat /var/run/webhcat/webhcat.pid`'```

Great.. thanks for the clarification.


- Robert


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53578/#review155322
---


On Nov. 8, 2016, 1:04 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53578/
> ---
> 
> (Updated Nov. 8, 2016, 1:04 p.m.)
> 
> 
> Review request for Ambari, Nate Cole and Robert Levas.
> 
> 
> Bugs: AMBARI-18824
> https://issues.apache.org/jira/browse/AMBARI-18824
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> There was code added in AMBARI-12695 (later corrected by AMBARI-18684) which 
> tries to kill WebHCat if it doesn't stop with its own stop command.
> 
> Both patches use the {{as_user}} function to wrap the call in a bash login:
> ```
> ps -p `su hcat -l -s /bin/bash -c 'cat /var/run/webhcat/webhcat.pid'` 
> >/dev/null 2>&1
> ```
> 
> This, however, causes problems where there is a MOTD:
> ```
> ERROR: Process ID list syntax error.
> * simple selection *  * selection by list *
> -A all processes  -C by command name
> -N negate selection   -G by real group ID (supports names)
> -a all w/ tty except session leaders  -U by real user ID (supports names)
> 
> ...
> 
> ```
> 
> That's not good - why is it giving us back a bad command syntax? Here's the 
> reason:
> ```
> [root@ijzs-c229todgsecha-r6-re1-3 ~]# su hcat -l -s /bin/bash -c 'cat 
> /var/run/webhcat/webhcat.pid'
> This is MOTD message :)
> 25672
> ```
> 
> *The MOTD is messing up the output from commands for the {{hcat}} user. 
> Because we rely on the error code of {{ps}}, the same non-zero code is used 
> for a bad command and a missing process ID*.
> 
> Since the PID is world-readable, we don't need to run the {{only_if}} checks 
> as a separate user.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat_service.py
>  fc6b933 
>   ambari-server/src/test/python/stacks/2.0.6/HIVE/test_webhcat_server.py 
> 0880a19 
> 
> Diff: https://reviews.apache.org/r/53578/diff/
> 
> 
> Testing
> ---
> 
> Tested with non-root agents and a MOTD
> 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 49.346 s
> [INFO] Finished at: 2016-11-08T12:49:12-05:00
> [INFO] Final Memory: 20M/439M
> [INFO] 
> 
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Re: Review Request 53583: Optionally force username from LDAP authentication data to be lowercase in Ambari


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53583/#review155354
---




ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
 (line 3745)


Would a Boolean.parseBoolean(...) be easier to read here?



ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AmbariLdapBindAuthenticator.java
 (line 80)


Out of curiousity, can LDAP contain non-latin characters which might not 
convert properly to lowercase?


- Jonathan Hurley


On Nov. 8, 2016, 2:09 p.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53583/
> ---
> 
> (Updated Nov. 8, 2016, 2:09 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Laszlo Puskas, Nate Cole, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-18813
> https://issues.apache.org/jira/browse/AMBARI-18813
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Optionally force username from LDAP authentication data to be lowercase in 
> Ambari based on LDAP import configuration.
> 
> In some cases the username declared in the relevant LDAP object is in all 
> uppercase characters when the local Hadoop cluster expects the username to be 
> all lowercase. As of Ambari 2.4.0, the username specified from the LDAP data 
> is used to override the username known to Ambari.  This overwritten data may 
> be in all uppercase characters, potentially breaking local username 
> conventions. 
> 
> To help this scenario, provide a configuration option to force the username 
> obtained from the LDAP object to be converted to all lowercase character. 
> 
> For example `authentication.ldap.username.forceLowercase`.
> 
> This optional configuration value is to default to false to maintain current 
> functionality.
> 
> 
> Diffs
> -
> 
>   ambari-server/docs/configuration/index.md 52b4744 
>   
> ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
>  3103b6d 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AmbariLdapBindAuthenticator.java
>  917471b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/LdapServerProperties.java
>  a3086693 
>   
> ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariLdapBindAuthenticatorTest.java
>  6e4b836 
> 
> Diff: https://reviews.apache.org/r/53583/diff/
> 
> 
> Testing
> ---
> 
> Manually tested...
> 
> # Local test results:
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 37:51.683s
> [INFO] Finished at: Tue Nov 08 10:51:17 EST 2016
> [INFO] Final Memory: 62M/834M
> [INFO] 
> 
> 
> # Jenkins test result: PENDING
> 
> 
> Thanks,
> 
> Robert Levas
> 
>

Re: Review Request 53578: MOTD Can Prevent WebHCat From Shutting Down



> On Nov. 8, 2016, 1:33 p.m., Robert Levas wrote:
> > ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat_service.py,
> >  lines 86-88
> > 
> >
> > If removing `as_user`, does this need to indicate the user to run as?
> 
> Jonathan Hurley wrote:
> So, the `as_user` was only for the `only_if`. I'm not changing the 
> `Execute` itself since that's not the bug here. Since PID is accessible to 
> the world, then we don't need to run as a user to execute the 
> `process_id_exists_command`

Since this is a hard kill command, running it using ambari-sudo.sh is correct:
```'ambari-sudo.sh kill -9 `cat /var/run/webhcat/webhcat.pid`'```


- Jonathan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53578/#review155322
---


On Nov. 8, 2016, 1:04 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53578/
> ---
> 
> (Updated Nov. 8, 2016, 1:04 p.m.)
> 
> 
> Review request for Ambari, Nate Cole and Robert Levas.
> 
> 
> Bugs: AMBARI-18824
> https://issues.apache.org/jira/browse/AMBARI-18824
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> There was code added in AMBARI-12695 (later corrected by AMBARI-18684) which 
> tries to kill WebHCat if it doesn't stop with its own stop command.
> 
> Both patches use the {{as_user}} function to wrap the call in a bash login:
> ```
> ps -p `su hcat -l -s /bin/bash -c 'cat /var/run/webhcat/webhcat.pid'` 
> >/dev/null 2>&1
> ```
> 
> This, however, causes problems where there is a MOTD:
> ```
> ERROR: Process ID list syntax error.
> * simple selection *  * selection by list *
> -A all processes  -C by command name
> -N negate selection   -G by real group ID (supports names)
> -a all w/ tty except session leaders  -U by real user ID (supports names)
> 
> ...
> 
> ```
> 
> That's not good - why is it giving us back a bad command syntax? Here's the 
> reason:
> ```
> [root@ijzs-c229todgsecha-r6-re1-3 ~]# su hcat -l -s /bin/bash -c 'cat 
> /var/run/webhcat/webhcat.pid'
> This is MOTD message :)
> 25672
> ```
> 
> *The MOTD is messing up the output from commands for the {{hcat}} user. 
> Because we rely on the error code of {{ps}}, the same non-zero code is used 
> for a bad command and a missing process ID*.
> 
> Since the PID is world-readable, we don't need to run the {{only_if}} checks 
> as a separate user.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat_service.py
>  fc6b933 
>   ambari-server/src/test/python/stacks/2.0.6/HIVE/test_webhcat_server.py 
> 0880a19 
> 
> Diff: https://reviews.apache.org/r/53578/diff/
> 
> 
> Testing
> ---
> 
> Tested with non-root agents and a MOTD
> 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 49.346 s
> [INFO] Finished at: 2016-11-08T12:49:12-05:00
> [INFO] Final Memory: 20M/439M
> [INFO] 
> 
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Re: Review Request 53578: MOTD Can Prevent WebHCat From Shutting Down



> On Nov. 8, 2016, 1:33 p.m., Robert Levas wrote:
> > ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat_service.py,
> >  lines 86-88
> > 
> >
> > If removing `as_user`, does this need to indicate the user to run as?

So, the `as_user` was only for the `only_if`. I'm not changing the `Execute` 
itself since that's not the bug here. Since PID is accessible to the world, 
then we don't need to run as a user to execute the `process_id_exists_command`


- Jonathan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53578/#review155322
---


On Nov. 8, 2016, 1:04 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53578/
> ---
> 
> (Updated Nov. 8, 2016, 1:04 p.m.)
> 
> 
> Review request for Ambari, Nate Cole and Robert Levas.
> 
> 
> Bugs: AMBARI-18824
> https://issues.apache.org/jira/browse/AMBARI-18824
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> There was code added in AMBARI-12695 (later corrected by AMBARI-18684) which 
> tries to kill WebHCat if it doesn't stop with its own stop command.
> 
> Both patches use the {{as_user}} function to wrap the call in a bash login:
> ```
> ps -p `su hcat -l -s /bin/bash -c 'cat /var/run/webhcat/webhcat.pid'` 
> >/dev/null 2>&1
> ```
> 
> This, however, causes problems where there is a MOTD:
> ```
> ERROR: Process ID list syntax error.
> * simple selection *  * selection by list *
> -A all processes  -C by command name
> -N negate selection   -G by real group ID (supports names)
> -a all w/ tty except session leaders  -U by real user ID (supports names)
> 
> ...
> 
> ```
> 
> That's not good - why is it giving us back a bad command syntax? Here's the 
> reason:
> ```
> [root@ijzs-c229todgsecha-r6-re1-3 ~]# su hcat -l -s /bin/bash -c 'cat 
> /var/run/webhcat/webhcat.pid'
> This is MOTD message :)
> 25672
> ```
> 
> *The MOTD is messing up the output from commands for the {{hcat}} user. 
> Because we rely on the error code of {{ps}}, the same non-zero code is used 
> for a bad command and a missing process ID*.
> 
> Since the PID is world-readable, we don't need to run the {{only_if}} checks 
> as a separate user.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat_service.py
>  fc6b933 
>   ambari-server/src/test/python/stacks/2.0.6/HIVE/test_webhcat_server.py 
> 0880a19 
> 
> Diff: https://reviews.apache.org/r/53578/diff/
> 
> 
> Testing
> ---
> 
> Tested with non-root agents and a MOTD
> 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 49.346 s
> [INFO] Finished at: 2016-11-08T12:49:12-05:00
> [INFO] Final Memory: 20M/439M
> [INFO] 
> 
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Re: Review Request 53565: Add ability to Skip Host check option on bootstrap page

2016-11-08 Thread Richard Zang


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53565/#review155349
---


Ship it!




Ship It!

- Richard Zang


On Nov. 8, 2016, 8:09 p.m., Jaimin Jetly wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53565/
> ---
> 
> (Updated Nov. 8, 2016, 8:09 p.m.)
> 
> 
> Review request for Ambari, Richard Zang, Vivek Ratnavel Subramanian, Xi Wang, 
> and Yusaku Sako.
> 
> 
> Bugs: AMBARI-18817
> https://issues.apache.org/jira/browse/AMBARI-18817
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> As part of this work experimental flag has been added by which host checks 
> can be disabled while going thorugh "Add Host Wizard"
> 
> 
> Diffs
> -
> 
>   ambari-web/app/config.js 06faf27 
>   ambari-web/app/controllers/wizard/step3_controller.js 0ccf2ed 
>   ambari-web/app/templates/wizard/step3.hbs f5648e1 
> 
> Diff: https://reviews.apache.org/r/53565/diff/
> 
> 
> Testing
> ---
> 
> manually verified the patch on the lcuster
> verified that all ambar-web unit tests passes
>   29248 tests complete (30 seconds)
>   154 tests pending
> 
> 
> Thanks,
> 
> Jaimin Jetly
> 
>

Review Request 53584: Ensure HAWQ -PXF service check uses a tmp directory

2016-11-08 Thread Matt


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53584/
---

Review request for Ambari and Lav Jain.


Bugs: AMBARI-18828
https://issues.apache.org/jira/browse/AMBARI-18828


Repository: ambari


Description
---

Ensure HAWQ -PXF service check uses a tmp directory and that this directory is 
deleted after the service check has been completed.


Diffs
-

  
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/hawq_constants.py
 3ced8d1 
  
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/service_check.py
 6e7ea0f 

Diff: https://reviews.apache.org/r/53584/diff/


Testing
---

Manually tested


Thanks,

Matt

Re: Review Request 53565: Add ability to Skip Host check option on bootstrap page

2016-11-08 Thread Vivek Ratnavel Subramanian


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53565/#review155343
---


Ship it!




Ship It!

- Vivek Ratnavel Subramanian


On Nov. 8, 2016, 8:09 p.m., Jaimin Jetly wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53565/
> ---
> 
> (Updated Nov. 8, 2016, 8:09 p.m.)
> 
> 
> Review request for Ambari, Richard Zang, Vivek Ratnavel Subramanian, Xi Wang, 
> and Yusaku Sako.
> 
> 
> Bugs: AMBARI-18817
> https://issues.apache.org/jira/browse/AMBARI-18817
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> As part of this work experimental flag has been added by which host checks 
> can be disabled while going thorugh "Add Host Wizard"
> 
> 
> Diffs
> -
> 
>   ambari-web/app/config.js 06faf27 
>   ambari-web/app/controllers/wizard/step3_controller.js 0ccf2ed 
>   ambari-web/app/templates/wizard/step3.hbs f5648e1 
> 
> Diff: https://reviews.apache.org/r/53565/diff/
> 
> 
> Testing
> ---
> 
> manually verified the patch on the lcuster
> verified that all ambar-web unit tests passes
>   29248 tests complete (30 seconds)
>   154 tests pending
> 
> 
> Thanks,
> 
> Jaimin Jetly
> 
>

Re: Review Request 53565: Add ability to Skip Host check option on bootstrap page

2016-11-08 Thread Jaimin Jetly



> On Nov. 8, 2016, 8 p.m., Vivek Ratnavel Subramanian wrote:
> > ambari-web/app/controllers/wizard/step3_controller.js, line 228
> > 
> >
> > Variable is not used

2nd version of the patch uploaded to fix this issue


- Jaimin


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53565/#review155339
---


On Nov. 8, 2016, 8:09 p.m., Jaimin Jetly wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53565/
> ---
> 
> (Updated Nov. 8, 2016, 8:09 p.m.)
> 
> 
> Review request for Ambari, Richard Zang, Vivek Ratnavel Subramanian, Xi Wang, 
> and Yusaku Sako.
> 
> 
> Bugs: AMBARI-18817
> https://issues.apache.org/jira/browse/AMBARI-18817
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> As part of this work experimental flag has been added by which host checks 
> can be disabled while going thorugh "Add Host Wizard"
> 
> 
> Diffs
> -
> 
>   ambari-web/app/config.js 06faf27 
>   ambari-web/app/controllers/wizard/step3_controller.js 0ccf2ed 
>   ambari-web/app/templates/wizard/step3.hbs f5648e1 
> 
> Diff: https://reviews.apache.org/r/53565/diff/
> 
> 
> Testing
> ---
> 
> manually verified the patch on the lcuster
> verified that all ambar-web unit tests passes
>   29248 tests complete (30 seconds)
>   154 tests pending
> 
> 
> Thanks,
> 
> Jaimin Jetly
> 
>

Re: Review Request 53565: Add ability to Skip Host check option on bootstrap page

2016-11-08 Thread Jaimin Jetly


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53565/
---

(Updated Nov. 8, 2016, 8:09 p.m.)


Review request for Ambari, Richard Zang, Vivek Ratnavel Subramanian, Xi Wang, 
and Yusaku Sako.


Changes
---

Removes an unused declared variable


Bugs: AMBARI-18817
https://issues.apache.org/jira/browse/AMBARI-18817


Repository: ambari


Description
---

As part of this work experimental flag has been added by which host checks can 
be disabled while going thorugh "Add Host Wizard"


Diffs (updated)
-

  ambari-web/app/config.js 06faf27 
  ambari-web/app/controllers/wizard/step3_controller.js 0ccf2ed 
  ambari-web/app/templates/wizard/step3.hbs f5648e1 

Diff: https://reviews.apache.org/r/53565/diff/


Testing
---

manually verified the patch on the lcuster
verified that all ambar-web unit tests passes
  29248 tests complete (30 seconds)
  154 tests pending


Thanks,

Jaimin Jetly

Re: Review Request 53565: Add ability to Skip Host check option on bootstrap page

2016-11-08 Thread Vivek Ratnavel Subramanian


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53565/#review155339
---




ambari-web/app/controllers/wizard/step3_controller.js (line 228)


Variable is not used


- Vivek Ratnavel Subramanian


On Nov. 8, 2016, 3:29 a.m., Jaimin Jetly wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53565/
> ---
> 
> (Updated Nov. 8, 2016, 3:29 a.m.)
> 
> 
> Review request for Ambari, Richard Zang, Vivek Ratnavel Subramanian, Xi Wang, 
> and Yusaku Sako.
> 
> 
> Bugs: AMBARI-18817
> https://issues.apache.org/jira/browse/AMBARI-18817
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> As part of this work experimental flag has been added by which host checks 
> can be disabled while going thorugh "Add Host Wizard"
> 
> 
> Diffs
> -
> 
>   ambari-web/app/config.js 06faf27 
>   ambari-web/app/controllers/wizard/step3_controller.js 0ccf2ed 
>   ambari-web/app/templates/wizard/step3.hbs f5648e1 
> 
> Diff: https://reviews.apache.org/r/53565/diff/
> 
> 
> Testing
> ---
> 
> manually verified the patch on the lcuster
> verified that all ambar-web unit tests passes
>   29248 tests complete (30 seconds)
>   154 tests pending
> 
> 
> Thanks,
> 
> Jaimin Jetly
> 
>

Re: Review Request 53496: Perf: Automation script to manage the lifecycle of multiple Ambari Agents on a single Host


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53496/
---

(Updated Nov. 8, 2016, 7:46 p.m.)


Review request for Ambari, Andrew Onischuk, Aravindan Vijayan, Myroslav 
Papirkovskyy, Sumit Mohanty, Sid Wagle, and Vitalyi Brodetskyi.


Bugs: AMBARI-18735
https://issues.apache.org/jira/browse/AMBARI-18735


Repository: ambari


Description
---

We need a method of easily bootstrapping multiple Ambari Agents on a single 
host.
For example, a script that will create each agent's home directory, modify its 
config file with unique hostname and port number, etc.
The script should also allow starting/stopping all agents on the host, and 
aggregating the status.


Diffs (updated)
-

  ambari-agent/conf/unix/agent-multiplier.py PRE-CREATION 

Diff: https://reviews.apache.org/r/53496/diff/


Testing
---

Verified it works locally and on a cloud environment.


Thanks,

Alejandro Fernandez

Re: Review Request 53564: Ranger install asks to populate AD domain and marks its required when no AD is in play

2016-11-08 Thread Jaimin Jetly



> On Nov. 8, 2016, 5:21 p.m., Jayush Luniya wrote:
> > ambari-web/test/utils/config_test.js, line 134
> > 
> >
> > The message is a bit misleading. HDF-2.0 is equivalent to HDP-2.5. The 
> > logic really is for which base stack folder to use. 
> > 
> > Also is there a long term solution to make this logic stack agnostic?

>> The message is a bit misleading. HDF-2.0 is equivalent to HDP-2.5. The logic 
>> really is for which base stack folder to use. 
we don't have any hard coded config property related definitions specific to 
HDP-2.4 and HDP-2.5 in ambari-web. so any stack greater than HDP-2.3 (HDP-2.4 
or HDP-2.5) or corresponding to HDP stack greater than HDP-2.3 (HDF-2.0) should 
be extending of HDP-2.3 stack definition

>> Also is there a long term solution to make this logic stack agnostic?
Ideal stack agnostic solution needs to be scoped and will definietely take more 
code changes which is not advisable to be done for immediate Ambari-2.4.2 
release. We can scope it for Ambari-3.0.0 (based on trunk). I have created 
https://issues.apache.org/jira/browse/AMBARI-18826 for that.


- Jaimin


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53564/#review155307
---


On Nov. 8, 2016, 2:45 a.m., Jaimin Jetly wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53564/
> ---
> 
> (Updated Nov. 8, 2016, 2:45 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya, Richard Zang, Vivek Ratnavel 
> Subramanian, Xi Wang, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-18816
> https://issues.apache.org/jira/browse/AMBARI-18816
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Ranger install asks to populate AD domain and marks its required when no AD 
> is in play
> 
> 
> Diffs
> -
> 
>   ambari-web/app/data/custom_stack_map.js 33b642c 
>   ambari-web/app/utils/config.js e83d42c 
>   ambari-web/test/utils/config_test.js 35d272d 
> 
> Diff: https://reviews.apache.org/r/53564/diff/
> 
> 
> Testing
> ---
> 
> Added new unit test.
> Verified that all existing unit tests passes:
> 
> 
>   29250 tests complete (29 seconds)
>   154 tests pending
> 
> 
> Thanks,
> 
> Jaimin Jetly
> 
>

Re: Review Request 53563: AMBARI-18815 - Manage Journal Node Wizard: Link Add/Delete operation for JournalNode to the wizard

2016-11-08 Thread Richard Zang



> On Nov. 8, 2016, 6:38 p.m., Alejandro Fernandez wrote:
> > ambari-web/app/views/main/host/details/host_component_view.js, line 205
> > 
> >
> > Will this TODO be fixed?

Yes, this will be fixed in following up bug fixing JIRAs.


- Richard


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53563/#review155323
---


On Nov. 8, 2016, 1:39 a.m., Richard Zang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53563/
> ---
> 
> (Updated Nov. 8, 2016, 1:39 a.m.)
> 
> 
> Review request for Ambari, Jaimin Jetly, Zhe (Joe) Wang, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-18815
> https://issues.apache.org/jira/browse/AMBARI-18815
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Link Add/Delete operation for JournalNode to the wizard
> 
> 
> Diffs
> -
> 
>   ambari-web/app/controllers/main/host/details.js 2076333 
>   ambari-web/app/messages.js d8ec229 
>   ambari-web/app/models/stack_service_component.js 17c2aa1 
>   ambari-web/app/templates/main/host/details/deleteComponentPopup.hbs 4fd5c52 
>   ambari-web/app/views/main/host/details/host_component_view.js 16071eb 
> 
> Diff: https://reviews.apache.org/r/53563/diff/
> 
> 
> Testing
> ---
> 
> Manually tested on live cluster.
> All unit tests passed.
>   25347 tests complete (21 seconds)
>   57 tests pending
> 
> 
> Thanks,
> 
> Richard Zang
> 
>

Review Request 53583: Optionally force username from LDAP authentication data to be lowercase in Ambari

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53583/
---

Review request for Ambari, Jonathan Hurley, Laszlo Puskas, Nate Cole, and
Sebastian Toader.

Bugs: AMBARI-18813
https://issues.apache.org/jira/browse/AMBARI-18813

Repository: ambari

Description
---

Optionally force username from LDAP authentication data to be lowercase in
Ambari based on LDAP import configuration.

In some cases the username declared in the relevant LDAP object is in all
uppercase characters when the local Hadoop cluster expects the username to be
all lowercase. As of Ambari 2.4.0, the username specified from the LDAP data is
used to override the username known to Ambari. This overwritten data may be in
all uppercase characters, potentially breaking local username conventions.

To help this scenario, provide a configuration option to force the username
obtained from the LDAP object to be converted to all lowercase character.

For example `authentication.ldap.username.forceLowercase`.

This optional configuration value is to default to false to maintain current
functionality.

Diffs
-

ambari-server/docs/configuration/index.md 52b4744

ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
3103b6d

ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AmbariLdapBindAuthenticator.java
917471b

ambari-server/src/main/java/org/apache/ambari/server/security/authorization/LdapServerProperties.java
a3086693

ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariLdapBindAuthenticatorTest.java
6e4b836

Diff: https://reviews.apache.org/r/53583/diff/

Testing
---

Manually tested...

# Local test results:
[INFO]
[INFO] BUILD SUCCESS
[INFO]
[INFO] Total time: 37:51.683s
[INFO] Finished at: Tue Nov 08 10:51:17 EST 2016
[INFO] Final Memory: 62M/834M
[INFO]

# Jenkins test result: PENDING

Thanks,

Robert Levas

Re: Review Request 53563: AMBARI-18815 - Manage Journal Node Wizard: Link Add/Delete operation for JournalNode to the wizard


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53563/#review155323
---




ambari-web/app/views/main/host/details/host_component_view.js (line 205)


Will this TODO be fixed?


- Alejandro Fernandez


On Nov. 8, 2016, 1:39 a.m., Richard Zang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53563/
> ---
> 
> (Updated Nov. 8, 2016, 1:39 a.m.)
> 
> 
> Review request for Ambari, Jaimin Jetly, Zhe (Joe) Wang, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-18815
> https://issues.apache.org/jira/browse/AMBARI-18815
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Link Add/Delete operation for JournalNode to the wizard
> 
> 
> Diffs
> -
> 
>   ambari-web/app/controllers/main/host/details.js 2076333 
>   ambari-web/app/messages.js d8ec229 
>   ambari-web/app/models/stack_service_component.js 17c2aa1 
>   ambari-web/app/templates/main/host/details/deleteComponentPopup.hbs 4fd5c52 
>   ambari-web/app/views/main/host/details/host_component_view.js 16071eb 
> 
> Diff: https://reviews.apache.org/r/53563/diff/
> 
> 
> Testing
> ---
> 
> Manually tested on live cluster.
> All unit tests passed.
>   25347 tests complete (21 seconds)
>   57 tests pending
> 
> 
> Thanks,
> 
> Richard Zang
> 
>

Re: Review Request 53578: MOTD Can Prevent WebHCat From Shutting Down


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53578/#review155322
---


Fix it, then Ship it!




Ship It!


ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat_service.py
 (lines 85 - 87)


If removing `as_user`, does this need to indicate the user to run as?


- Robert Levas


On Nov. 8, 2016, 1:04 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53578/
> ---
> 
> (Updated Nov. 8, 2016, 1:04 p.m.)
> 
> 
> Review request for Ambari, Nate Cole and Robert Levas.
> 
> 
> Bugs: AMBARI-18824
> https://issues.apache.org/jira/browse/AMBARI-18824
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> There was code added in AMBARI-12695 (later corrected by AMBARI-18684) which 
> tries to kill WebHCat if it doesn't stop with its own stop command.
> 
> Both patches use the {{as_user}} function to wrap the call in a bash login:
> ```
> ps -p `su hcat -l -s /bin/bash -c 'cat /var/run/webhcat/webhcat.pid'` 
> >/dev/null 2>&1
> ```
> 
> This, however, causes problems where there is a MOTD:
> ```
> ERROR: Process ID list syntax error.
> * simple selection *  * selection by list *
> -A all processes  -C by command name
> -N negate selection   -G by real group ID (supports names)
> -a all w/ tty except session leaders  -U by real user ID (supports names)
> 
> ...
> 
> ```
> 
> That's not good - why is it giving us back a bad command syntax? Here's the 
> reason:
> ```
> [root@ijzs-c229todgsecha-r6-re1-3 ~]# su hcat -l -s /bin/bash -c 'cat 
> /var/run/webhcat/webhcat.pid'
> This is MOTD message :)
> 25672
> ```
> 
> *The MOTD is messing up the output from commands for the {{hcat}} user. 
> Because we rely on the error code of {{ps}}, the same non-zero code is used 
> for a bad command and a missing process ID*.
> 
> Since the PID is world-readable, we don't need to run the {{only_if}} checks 
> as a separate user.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat_service.py
>  fc6b933 
>   ambari-server/src/test/python/stacks/2.0.6/HIVE/test_webhcat_server.py 
> 0880a19 
> 
> Diff: https://reviews.apache.org/r/53578/diff/
> 
> 
> Testing
> ---
> 
> Tested with non-root agents and a MOTD
> 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 49.346 s
> [INFO] Finished at: 2016-11-08T12:49:12-05:00
> [INFO] Final Memory: 20M/439M
> [INFO] 
> 
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Re: Review Request 53575: Upgrade from 2.4.2 to 2.5 fails with DB consistency check


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53575/#review155320
---


Ship it!




Ship It!

- Alejandro Fernandez


On Nov. 8, 2016, 4:47 p.m., Vitalyi Brodetskyi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53575/
> ---
> 
> (Updated Nov. 8, 2016, 4:47 p.m.)
> 
> 
> Review request for Ambari, Dmitro Lisnichenko, Dmytro Sen, Sumit Mohanty, and 
> Sid Wagle.
> 
> 
> Bugs: AMBARI-18822
> https://issues.apache.org/jira/browse/AMBARI-18822
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> {code}
> 18:03:32,959 ERROR - Required config(s): 
> hdfs-logsearch-conf,hadoop-metrics2.properties is(are) not available for 
> service HDFS with service config version 1 in cluster cl1
> 2016-11-07 18:03:32,959 ERROR - Required config(s): mapred-logsearch-conf 
> is(are) not available for service MAPREDUCE2 with service config version 1 in 
> cluster cl1
> 2016-11-07 18:03:32,959 ERROR - Required config(s): zookeeper-logsearch-conf 
> is(are) not available for service ZOOKEEPER with service config version 1 in 
> cluster cl1
> 2016-11-07 18:03:32,960 ERROR - Required config(s): yarn-logsearch-conf 
> is(are) not available for service YARN with service config version 1 in 
> cluster cl1
> 2016-11-07 18:03:32,960 ERROR - Required config(s): ams-logsearch-conf 
> is(are) not available for service AMBARI_METRICS with service config version 
> 2 in cluster cl1
> {code}
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
>  a7d5af9 
>   
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog250Test.java
>  5ac0822 
> 
> Diff: https://reviews.apache.org/r/53575/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>

Re: Review Request 53576: AMBARI-18569 Execute topology tasks in parallel by hosts


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53576/#review155319
---


Ship it!




Ship It!

- Alejandro Fernandez


On Nov. 8, 2016, 5:14 p.m., Attila Doroszlai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53576/
> ---
> 
> (Updated Nov. 8, 2016, 5:14 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Sandor Magyari, and Sebastian 
> Toader.
> 
> 
> Bugs: AMBARI-18569
> https://issues.apache.org/jira/browse/AMBARI-18569
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> * Configurable number of threads
>* Enable/disable flag
>* Removed synchronization at bottlenecks
>* Added cache for `StackEntity.findByNameAndVersion` query
>* Added log for all 4 types of topology tasks
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
>  3103b6d 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  09e49ef 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ComponentResourceProvider.java
>  241a48f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostComponentResourceProvider.java
>  87eb266 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ServiceResourceProvider.java
>  a08d153 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/StackEntity.java
>  c425969 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/HostOfferResponse.java
>  2932581 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/HostRequest.java
>  6a65b48 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/LogicalRequest.java
>  3aaf589 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/TopologyManager.java
>  bba0325 
> 
> Diff: https://reviews.apache.org/r/53576/diff/
> 
> 
> Testing
> ---
> 
> Created several clusters (size = 8-64 nodes) via blueprint
> 
> 
> Thanks,
> 
> Attila Doroszlai
> 
>

Review Request 53578: MOTD Can Prevent WebHCat From Shutting Down


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53578/
---

Review request for Ambari, Nate Cole and Robert Levas.


Bugs: AMBARI-18824
https://issues.apache.org/jira/browse/AMBARI-18824


Repository: ambari


Description
---

There was code added in AMBARI-12695 (later corrected by AMBARI-18684) which 
tries to kill WebHCat if it doesn't stop with its own stop command.

Both patches use the {{as_user}} function to wrap the call in a bash login:
```
ps -p `su hcat -l -s /bin/bash -c 'cat /var/run/webhcat/webhcat.pid'` 
>/dev/null 2>&1
```

This, however, causes problems where there is a MOTD:
```
ERROR: Process ID list syntax error.
* simple selection *  * selection by list *
-A all processes  -C by command name
-N negate selection   -G by real group ID (supports names)
-a all w/ tty except session leaders  -U by real user ID (supports names)

...

```

That's not good - why is it giving us back a bad command syntax? Here's the 
reason:
```
[root@ijzs-c229todgsecha-r6-re1-3 ~]# su hcat -l -s /bin/bash -c 'cat 
/var/run/webhcat/webhcat.pid'
This is MOTD message :)
25672
```

*The MOTD is messing up the output from commands for the {{hcat}} user. Because 
we rely on the error code of {{ps}}, the same non-zero code is used for a bad 
command and a missing process ID*.

Since the PID is world-readable, we don't need to run the {{only_if}} checks as 
a separate user.


Diffs
-

  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat_service.py
 fc6b933 
  ambari-server/src/test/python/stacks/2.0.6/HIVE/test_webhcat_server.py 
0880a19 

Diff: https://reviews.apache.org/r/53578/diff/


Testing
---

Tested with non-root agents and a MOTD

[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 49.346 s
[INFO] Finished at: 2016-11-08T12:49:12-05:00
[INFO] Final Memory: 20M/439M
[INFO] 


Thanks,

Jonathan Hurley

Re: Review Request 53576: AMBARI-18569 Execute topology tasks in parallel by hosts

2016-11-08 Thread Sandor Magyari


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53576/#review155316
---


Ship it!




Ship It!

- Sandor Magyari


On Nov. 8, 2016, 5:14 p.m., Attila Doroszlai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53576/
> ---
> 
> (Updated Nov. 8, 2016, 5:14 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Sandor Magyari, and Sebastian 
> Toader.
> 
> 
> Bugs: AMBARI-18569
> https://issues.apache.org/jira/browse/AMBARI-18569
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> * Configurable number of threads
>* Enable/disable flag
>* Removed synchronization at bottlenecks
>* Added cache for `StackEntity.findByNameAndVersion` query
>* Added log for all 4 types of topology tasks
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
>  3103b6d 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  09e49ef 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ComponentResourceProvider.java
>  241a48f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostComponentResourceProvider.java
>  87eb266 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ServiceResourceProvider.java
>  a08d153 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/StackEntity.java
>  c425969 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/HostOfferResponse.java
>  2932581 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/HostRequest.java
>  6a65b48 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/LogicalRequest.java
>  3aaf589 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/TopologyManager.java
>  bba0325 
> 
> Diff: https://reviews.apache.org/r/53576/diff/
> 
> 
> Testing
> ---
> 
> Created several clusters (size = 8-64 nodes) via blueprint
> 
> 
> Thanks,
> 
> Attila Doroszlai
> 
>

Re: Review Request 53575: Upgrade from 2.4.2 to 2.5 fails with DB consistency check

2016-11-08 Thread Dmitro Lisnichenko


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53575/#review155311
---


Ship it!




Ship It!

- Dmitro Lisnichenko


On Nov. 8, 2016, 6:47 p.m., Vitalyi Brodetskyi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53575/
> ---
> 
> (Updated Nov. 8, 2016, 6:47 p.m.)
> 
> 
> Review request for Ambari, Dmitro Lisnichenko, Dmytro Sen, Sumit Mohanty, and 
> Sid Wagle.
> 
> 
> Bugs: AMBARI-18822
> https://issues.apache.org/jira/browse/AMBARI-18822
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> {code}
> 18:03:32,959 ERROR - Required config(s): 
> hdfs-logsearch-conf,hadoop-metrics2.properties is(are) not available for 
> service HDFS with service config version 1 in cluster cl1
> 2016-11-07 18:03:32,959 ERROR - Required config(s): mapred-logsearch-conf 
> is(are) not available for service MAPREDUCE2 with service config version 1 in 
> cluster cl1
> 2016-11-07 18:03:32,959 ERROR - Required config(s): zookeeper-logsearch-conf 
> is(are) not available for service ZOOKEEPER with service config version 1 in 
> cluster cl1
> 2016-11-07 18:03:32,960 ERROR - Required config(s): yarn-logsearch-conf 
> is(are) not available for service YARN with service config version 1 in 
> cluster cl1
> 2016-11-07 18:03:32,960 ERROR - Required config(s): ams-logsearch-conf 
> is(are) not available for service AMBARI_METRICS with service config version 
> 2 in cluster cl1
> {code}
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
>  a7d5af9 
>   
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog250Test.java
>  5ac0822 
> 
> Diff: https://reviews.apache.org/r/53575/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>

Re: Review Request 53454: AMBARI-18795: Ambari-server: Stack definition to specify which services support credential store

2016-11-08 Thread Jayush Luniya


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53454/#review155310
---


Ship it!




Ship It!

- Jayush Luniya


On Nov. 8, 2016, 5:16 p.m., Nahappan Somasundaram wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53454/
> ---
> 
> (Updated Nov. 8, 2016, 5:16 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Jayush Luniya, Nate Cole, Robert 
> Levas, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-18795
> https://issues.apache.org/jira/browse/AMBARI-18795
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> AMBARI-18795: Ambari-server: Stack definition to specify which services 
> support credential store
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER/metainfo.xml 
> c5a727ca549af8accea1eceee59bb9662785146e 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/services/RANGER/metainfo.xml 
> 811c5e9b443b1d13bdc1884d098d0ad726f92f0d 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/metainfo.xml 
> a409465cbf8e6a457f8330c7a7bf073bdbb716e6 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/KNOX/metainfo.xml 
> 8f303a54a5128520bea23f6f6c87ad7ae41965ed 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/RANGER/metainfo.xml 
> fe97a41657dcd647707729951a4d09fa5eb870e5 
> 
> Diff: https://reviews.apache.org/r/53454/diff/
> 
> 
> Testing
> ---
> 
> ** Manual testing **
> 
> Copied the modified stack definition files to a VM, deployed RANGER, HIVE and 
> KNOX for HDP-2.5. Verified in the database that the following entries were 
> made:
> 
> ambari=> select service_name, credential_store_supported, 
> credential_store_enabled from servicedesiredstate;
> 
>  service_name | credential_store_supported | credential_store_enabled
> --++--
>  HDFS |  0 |0
>  HIVE |  1 |0
>  KNOX |  1 |0
>  MAPREDUCE2   |  0 |0
>  PIG  |  0 |0
>  SLIDER   |  0 |0
>  TEZ  |  0 |0
>  YARN |  0 |0
>  ZOOKEEPER|  0 |0
>  RANGER   |  1 |1
> 
> 
> Thanks,
> 
> Nahappan Somasundaram
> 
>

Re: Review Request 53502: AMBARI-18681 Add host structured_out to manual task

2016-11-08 Thread Duc Le



> On Nov. 8, 2016, 3:24 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/HostOrderGrouping.java,
> >  line 179
> > 
> >
> > I did mark it as Ship It, but I'm curious if we want to have this as a 
> > separate enum?
> 
> Duc Le wrote:
> If we don't use HostOrderActionType.HOST_UPGRADE, I would use a constant 
> string for now until we know better about what possible values of the enum 
> are.
> 
> Jonathan Hurley wrote:
> I think the original idea was to give a clue about why the upgrade was 
> holding and what needed to be done. HOST_UPGRADE is probably fine for this 
> (the original value in the Jira was `HOST_PREPARE` I think).
> 
> Nate Cole wrote:
> @Duc, when you're ready, I can commit this on your behalf.  Just ping me 
> when ready.

@Nate, this should be ready, so please commit this change. I still can't update 
Jiras/attach patches yet. Thanks!


- Duc


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53502/#review155291
---


On Nov. 8, 2016, 3:42 a.m., Duc Le wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53502/
> ---
> 
> (Updated Nov. 8, 2016, 3:42 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, and Nate 
> Cole.
> 
> 
> Bugs: AMBARI-18681
> https://issues.apache.org/jira/browse/AMBARI-18681
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> AMBARI-18681 Add host structured_out to manual task
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/HostOrderGrouping.java
>  b999f2f 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/UpgradeHelperTest.java
>  78969e4 
> 
> Diff: https://reviews.apache.org/r/53502/diff/
> 
> 
> Testing
> ---
> 
> Unit tests passing.
> 
> ...
> Tests run: 28, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 209.464 sec 
> - in org.apache.ambari.server.state.UpgradeHelperTest
> ...
> Tests run: 4727, Failures: 0, Errors: 0, Skipped: 42
> 
> 
> Thanks,
> 
> Duc Le
> 
>

Review Request 53576: AMBARI-18569 Execute topology tasks in parallel by hosts

2016-11-08 Thread Attila Doroszlai


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53576/
---

Review request for Ambari, Jonathan Hurley, Sandor Magyari, and Sebastian 
Toader.


Bugs: AMBARI-18569
https://issues.apache.org/jira/browse/AMBARI-18569


Repository: ambari


Description
---

* Configurable number of threads
   * Enable/disable flag
   * Removed synchronization at bottlenecks
   * Added cache for `StackEntity.findByNameAndVersion` query
   * Added log for all 4 types of topology tasks


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
 3103b6d 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
 09e49ef 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ComponentResourceProvider.java
 241a48f 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostComponentResourceProvider.java
 87eb266 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ServiceResourceProvider.java
 a08d153 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/StackEntity.java
 c425969 
  
ambari-server/src/main/java/org/apache/ambari/server/topology/HostOfferResponse.java
 2932581 
  
ambari-server/src/main/java/org/apache/ambari/server/topology/HostRequest.java 
6a65b48 
  
ambari-server/src/main/java/org/apache/ambari/server/topology/LogicalRequest.java
 3aaf589 
  
ambari-server/src/main/java/org/apache/ambari/server/topology/TopologyManager.java
 bba0325 

Diff: https://reviews.apache.org/r/53576/diff/


Testing
---

Created several clusters (size = 8-64 nodes) via blueprint


Thanks,

Attila Doroszlai

Re: Review Request 53564: Ranger install asks to populate AD domain and marks its required when no AD is in play

2016-11-08 Thread Jayush Luniya


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53564/#review155307
---




ambari-web/test/utils/config_test.js (line 134)


The message is a bit misleading. HDF-2.0 is equivalent to HDP-2.5. The 
logic really is for which base stack folder to use. 

Also is there a long term solution to make this logic stack agnostic?


- Jayush Luniya


On Nov. 8, 2016, 2:45 a.m., Jaimin Jetly wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53564/
> ---
> 
> (Updated Nov. 8, 2016, 2:45 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya, Richard Zang, Vivek Ratnavel 
> Subramanian, Xi Wang, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-18816
> https://issues.apache.org/jira/browse/AMBARI-18816
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Ranger install asks to populate AD domain and marks its required when no AD 
> is in play
> 
> 
> Diffs
> -
> 
>   ambari-web/app/data/custom_stack_map.js 33b642c 
>   ambari-web/app/utils/config.js e83d42c 
>   ambari-web/test/utils/config_test.js 35d272d 
> 
> Diff: https://reviews.apache.org/r/53564/diff/
> 
> 
> Testing
> ---
> 
> Added new unit test.
> Verified that all existing unit tests passes:
> 
> 
>   29250 tests complete (29 seconds)
>   154 tests pending
> 
> 
> Thanks,
> 
> Jaimin Jetly
> 
>

Re: Review Request 53454: AMBARI-18795: Ambari-server: Stack definition to specify which services support credential store

2016-11-08 Thread Nahappan Somasundaram


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53454/
---

(Updated Nov. 8, 2016, 9:16 a.m.)


Review request for Ambari, Jonathan Hurley, Jayush Luniya, Nate Cole, Robert 
Levas, and Sumit Mohanty.


Changes
---

Have RANGER in disable state. Enabling it now will cause issues with current 
RANGER scripts. Will be enabled when making changes to RANGER scripts for 
backward compatibility.


Bugs: AMBARI-18795
https://issues.apache.org/jira/browse/AMBARI-18795


Repository: ambari


Description
---

AMBARI-18795: Ambari-server: Stack definition to specify which services support 
credential store


Diffs (updated)
-

  ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER/metainfo.xml 
c5a727ca549af8accea1eceee59bb9662785146e 
  ambari-server/src/main/resources/stacks/HDP/2.4/services/RANGER/metainfo.xml 
811c5e9b443b1d13bdc1884d098d0ad726f92f0d 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/metainfo.xml 
a409465cbf8e6a457f8330c7a7bf073bdbb716e6 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/KNOX/metainfo.xml 
8f303a54a5128520bea23f6f6c87ad7ae41965ed 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/RANGER/metainfo.xml 
fe97a41657dcd647707729951a4d09fa5eb870e5 

Diff: https://reviews.apache.org/r/53454/diff/


Testing
---

** Manual testing **

Copied the modified stack definition files to a VM, deployed RANGER, HIVE and 
KNOX for HDP-2.5. Verified in the database that the following entries were made:

ambari=> select service_name, credential_store_supported, 
credential_store_enabled from servicedesiredstate;

 service_name | credential_store_supported | credential_store_enabled
--++--
 HDFS |  0 |0
 HIVE |  1 |0
 KNOX |  1 |0
 MAPREDUCE2   |  0 |0
 PIG  |  0 |0
 SLIDER   |  0 |0
 TEZ  |  0 |0
 YARN |  0 |0
 ZOOKEEPER|  0 |0
 RANGER   |  1 |1


Thanks,

Nahappan Somasundaram

Re: Review Request 53496: Perf: Automation script to manage the lifecycle of multiple Ambari Agents on a single Host

2016-11-08 Thread Vitalyi Brodetskyi


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53496/#review155303
---


Ship it!




Ship It!

- Vitalyi Brodetskyi


On Лис. 4, 2016, 6:27 після полудня, Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53496/
> ---
> 
> (Updated Лис. 4, 2016, 6:27 після полудня)
> 
> 
> Review request for Ambari, Andrew Onischuk, Aravindan Vijayan, Myroslav 
> Papirkovskyy, Sumit Mohanty, Sid Wagle, and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-18735
> https://issues.apache.org/jira/browse/AMBARI-18735
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> We need a method of easily bootstrapping multiple Ambari Agents on a single 
> host.
> For example, a script that will create each agent's home directory, modify 
> its config file with unique hostname and port number, etc.
> The script should also allow starting/stopping all agents on the host, and 
> aggregating the status.
> 
> 
> Diffs
> -
> 
>   ambari-agent/conf/unix/agent-multiplier.py PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/53496/diff/
> 
> 
> Testing
> ---
> 
> Verified it works locally and on a cloud environment.
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Re: Review Request 53273: Not able to change the default port ambari server listens to

2016-11-08 Thread Di Li



> On Nov. 8, 2016, 4:40 p.m., Di Li wrote:
> > Hello Anita, 
> > 
> > I pushed the change to trunk as 
> > https://git-wip-us.apache.org/repos/asf?p=ambari.git;a=commit;h=352fe85181d81cf23f82953f05bc66a7c88d639b
> > 
> > Please provide a patch rebased for branch-2.5, AMBARI-18741-Nov7.patch does 
> > not apply to branch-2.5
> 
> Anita Jebaraj wrote:
> Hi Di, these changes are not included in branch-2.5, this issue was 
> existing only in trunk.
> 
> Anita Jebaraj wrote:
> Thank you for pushing the changes

ok, good to know, please go ahead and close the RB request.


- Di


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53273/#review155299
---


On Nov. 7, 2016, 6:11 p.m., Anita Jebaraj wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53273/
> ---
> 
> (Updated Nov. 7, 2016, 6:11 p.m.)
> 
> 
> Review request for Ambari, Di Li, Tim Thorpe, and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-18741
> https://issues.apache.org/jira/browse/AMBARI-18741
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> After including client.api.port in ambari.properties, ambari-server still 
> tries to connect to port 8080
> 
> The ambari_server_ui_port is hardcoded to 8080, but if the client.api.port is 
> defined in ambari.properties then the value should be taken as the port 
> number.
> 
> https://docs.hortonworks.com/HDPDocuments/Ambari-2.1.1.0/bk_ambari_reference_guide/content/_optional_changing_the_default_ambari_server_port.html
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/python/ambari_server/serverConfiguration.py 766f4cc 
>   ambari-server/src/main/python/ambari_server/utils.py 757c333 
>   ambari-server/src/test/python/TestUtils.py 367b7fe 
> 
> Diff: https://reviews.apache.org/r/53273/diff/
> 
> 
> Testing
> ---
> 
> Performed manual testing.
> 
> 
> Thanks,
> 
> Anita Jebaraj
> 
>

Re: Review Request 53273: Not able to change the default port ambari server listens to

2016-11-08 Thread Anita Jebaraj



> On Nov. 8, 2016, 4:40 p.m., Di Li wrote:
> > Hello Anita, 
> > 
> > I pushed the change to trunk as 
> > https://git-wip-us.apache.org/repos/asf?p=ambari.git;a=commit;h=352fe85181d81cf23f82953f05bc66a7c88d639b
> > 
> > Please provide a patch rebased for branch-2.5, AMBARI-18741-Nov7.patch does 
> > not apply to branch-2.5
> 
> Anita Jebaraj wrote:
> Hi Di, these changes are not included in branch-2.5, this issue was 
> existing only in trunk.

Thank you for pushing the changes


- Anita


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53273/#review155299
---


On Nov. 7, 2016, 6:11 p.m., Anita Jebaraj wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53273/
> ---
> 
> (Updated Nov. 7, 2016, 6:11 p.m.)
> 
> 
> Review request for Ambari, Di Li, Tim Thorpe, and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-18741
> https://issues.apache.org/jira/browse/AMBARI-18741
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> After including client.api.port in ambari.properties, ambari-server still 
> tries to connect to port 8080
> 
> The ambari_server_ui_port is hardcoded to 8080, but if the client.api.port is 
> defined in ambari.properties then the value should be taken as the port 
> number.
> 
> https://docs.hortonworks.com/HDPDocuments/Ambari-2.1.1.0/bk_ambari_reference_guide/content/_optional_changing_the_default_ambari_server_port.html
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/python/ambari_server/serverConfiguration.py 766f4cc 
>   ambari-server/src/main/python/ambari_server/utils.py 757c333 
>   ambari-server/src/test/python/TestUtils.py 367b7fe 
> 
> Diff: https://reviews.apache.org/r/53273/diff/
> 
> 
> Testing
> ---
> 
> Performed manual testing.
> 
> 
> Thanks,
> 
> Anita Jebaraj
> 
>

Re: Review Request 53273: Not able to change the default port ambari server listens to

2016-11-08 Thread Anita Jebaraj



> On Nov. 8, 2016, 4:40 p.m., Di Li wrote:
> > Hello Anita, 
> > 
> > I pushed the change to trunk as 
> > https://git-wip-us.apache.org/repos/asf?p=ambari.git;a=commit;h=352fe85181d81cf23f82953f05bc66a7c88d639b
> > 
> > Please provide a patch rebased for branch-2.5, AMBARI-18741-Nov7.patch does 
> > not apply to branch-2.5

Hi Di, these changes are not included in branch-2.5, this issue was existing 
only in trunk.


- Anita


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53273/#review155299
---


On Nov. 7, 2016, 6:11 p.m., Anita Jebaraj wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53273/
> ---
> 
> (Updated Nov. 7, 2016, 6:11 p.m.)
> 
> 
> Review request for Ambari, Di Li, Tim Thorpe, and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-18741
> https://issues.apache.org/jira/browse/AMBARI-18741
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> After including client.api.port in ambari.properties, ambari-server still 
> tries to connect to port 8080
> 
> The ambari_server_ui_port is hardcoded to 8080, but if the client.api.port is 
> defined in ambari.properties then the value should be taken as the port 
> number.
> 
> https://docs.hortonworks.com/HDPDocuments/Ambari-2.1.1.0/bk_ambari_reference_guide/content/_optional_changing_the_default_ambari_server_port.html
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/python/ambari_server/serverConfiguration.py 766f4cc 
>   ambari-server/src/main/python/ambari_server/utils.py 757c333 
>   ambari-server/src/test/python/TestUtils.py 367b7fe 
> 
> Diff: https://reviews.apache.org/r/53273/diff/
> 
> 
> Testing
> ---
> 
> Performed manual testing.
> 
> 
> Thanks,
> 
> Anita Jebaraj
> 
>

Review Request 53575: Upgrade from 2.4.2 to 2.5 fails with DB consistency check

2016-11-08 Thread Vitalyi Brodetskyi


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53575/
---

Review request for Ambari, Dmitro Lisnichenko, Dmytro Sen, Sumit Mohanty, and 
Sid Wagle.


Bugs: AMBARI-18822
https://issues.apache.org/jira/browse/AMBARI-18822


Repository: ambari


Description
---

{code}
18:03:32,959 ERROR - Required config(s): 
hdfs-logsearch-conf,hadoop-metrics2.properties is(are) not available for 
service HDFS with service config version 1 in cluster cl1
2016-11-07 18:03:32,959 ERROR - Required config(s): mapred-logsearch-conf 
is(are) not available for service MAPREDUCE2 with service config version 1 in 
cluster cl1
2016-11-07 18:03:32,959 ERROR - Required config(s): zookeeper-logsearch-conf 
is(are) not available for service ZOOKEEPER with service config version 1 in 
cluster cl1
2016-11-07 18:03:32,960 ERROR - Required config(s): yarn-logsearch-conf is(are) 
not available for service YARN with service config version 1 in cluster cl1
2016-11-07 18:03:32,960 ERROR - Required config(s): ams-logsearch-conf is(are) 
not available for service AMBARI_METRICS with service config version 2 in 
cluster cl1
{code}


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
 a7d5af9 
  
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog250Test.java
 5ac0822 

Diff: https://reviews.apache.org/r/53575/diff/


Testing
---

mvn clean test


Thanks,

Vitalyi Brodetskyi

Re: Review Request 53273: Not able to change the default port ambari server listens to

2016-11-08 Thread Di Li


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53273/#review155299
---



Hello Anita, 

I pushed the change to trunk as 
https://git-wip-us.apache.org/repos/asf?p=ambari.git;a=commit;h=352fe85181d81cf23f82953f05bc66a7c88d639b

Please provide a patch rebased for branch-2.5, AMBARI-18741-Nov7.patch does not 
apply to branch-2.5

- Di Li


On Nov. 7, 2016, 6:11 p.m., Anita Jebaraj wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53273/
> ---
> 
> (Updated Nov. 7, 2016, 6:11 p.m.)
> 
> 
> Review request for Ambari, Di Li, Tim Thorpe, and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-18741
> https://issues.apache.org/jira/browse/AMBARI-18741
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> After including client.api.port in ambari.properties, ambari-server still 
> tries to connect to port 8080
> 
> The ambari_server_ui_port is hardcoded to 8080, but if the client.api.port is 
> defined in ambari.properties then the value should be taken as the port 
> number.
> 
> https://docs.hortonworks.com/HDPDocuments/Ambari-2.1.1.0/bk_ambari_reference_guide/content/_optional_changing_the_default_ambari_server_port.html
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/python/ambari_server/serverConfiguration.py 766f4cc 
>   ambari-server/src/main/python/ambari_server/utils.py 757c333 
>   ambari-server/src/test/python/TestUtils.py 367b7fe 
> 
> Diff: https://reviews.apache.org/r/53273/diff/
> 
> 
> Testing
> ---
> 
> Performed manual testing.
> 
> 
> Thanks,
> 
> Anita Jebaraj
> 
>

Re: Review Request 53502: AMBARI-18681 Add host structured_out to manual task



> On Nov. 8, 2016, 10:24 a.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/HostOrderGrouping.java,
> >  line 179
> > 
> >
> > I did mark it as Ship It, but I'm curious if we want to have this as a 
> > separate enum?
> 
> Duc Le wrote:
> If we don't use HostOrderActionType.HOST_UPGRADE, I would use a constant 
> string for now until we know better about what possible values of the enum 
> are.
> 
> Jonathan Hurley wrote:
> I think the original idea was to give a clue about why the upgrade was 
> holding and what needed to be done. HOST_UPGRADE is probably fine for this 
> (the original value in the Jira was `HOST_PREPARE` I think).

@Duc, when you're ready, I can commit this on your behalf.  Just ping me when 
ready.


- Nate


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53502/#review155291
---


On Nov. 7, 2016, 10:42 p.m., Duc Le wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53502/
> ---
> 
> (Updated Nov. 7, 2016, 10:42 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, and Nate 
> Cole.
> 
> 
> Bugs: AMBARI-18681
> https://issues.apache.org/jira/browse/AMBARI-18681
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> AMBARI-18681 Add host structured_out to manual task
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/HostOrderGrouping.java
>  b999f2f 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/UpgradeHelperTest.java
>  78969e4 
> 
> Diff: https://reviews.apache.org/r/53502/diff/
> 
> 
> Testing
> ---
> 
> Unit tests passing.
> 
> ...
> Tests run: 28, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 209.464 sec 
> - in org.apache.ambari.server.state.UpgradeHelperTest
> ...
> Tests run: 4727, Failures: 0, Errors: 0, Skipped: 42
> 
> 
> Thanks,
> 
> Duc Le
> 
>

Re: Review Request 53502: AMBARI-18681 Add host structured_out to manual task



> On Nov. 8, 2016, 10:24 a.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/HostOrderGrouping.java,
> >  line 179
> > 
> >
> > I did mark it as Ship It, but I'm curious if we want to have this as a 
> > separate enum?
> 
> Duc Le wrote:
> If we don't use HostOrderActionType.HOST_UPGRADE, I would use a constant 
> string for now until we know better about what possible values of the enum 
> are.

I think the original idea was to give a clue about why the upgrade was holding 
and what needed to be done. HOST_UPGRADE is probably fine for this (the 
original value in the Jira was `HOST_PREPARE` I think).


- Jonathan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53502/#review155291
---


On Nov. 7, 2016, 10:42 p.m., Duc Le wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53502/
> ---
> 
> (Updated Nov. 7, 2016, 10:42 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, and Nate 
> Cole.
> 
> 
> Bugs: AMBARI-18681
> https://issues.apache.org/jira/browse/AMBARI-18681
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> AMBARI-18681 Add host structured_out to manual task
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/HostOrderGrouping.java
>  b999f2f 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/UpgradeHelperTest.java
>  78969e4 
> 
> Diff: https://reviews.apache.org/r/53502/diff/
> 
> 
> Testing
> ---
> 
> Unit tests passing.
> 
> ...
> Tests run: 28, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 209.464 sec 
> - in org.apache.ambari.server.state.UpgradeHelperTest
> ...
> Tests run: 4727, Failures: 0, Errors: 0, Skipped: 42
> 
> 
> Thanks,
> 
> Duc Le
> 
>

Re: Review Request 53502: AMBARI-18681 Add host structured_out to manual task

2016-11-08 Thread Duc Le



> On Nov. 8, 2016, 3:24 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/HostOrderGrouping.java,
> >  line 179
> > 
> >
> > I did mark it as Ship It, but I'm curious if we want to have this as a 
> > separate enum?

If we don't use HostOrderActionType.HOST_UPGRADE, I would use a constant string 
for now until we know better about what possible values of the enum are.


- Duc


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53502/#review155291
---


On Nov. 8, 2016, 3:42 a.m., Duc Le wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53502/
> ---
> 
> (Updated Nov. 8, 2016, 3:42 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, and Nate 
> Cole.
> 
> 
> Bugs: AMBARI-18681
> https://issues.apache.org/jira/browse/AMBARI-18681
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> AMBARI-18681 Add host structured_out to manual task
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/HostOrderGrouping.java
>  b999f2f 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/UpgradeHelperTest.java
>  78969e4 
> 
> Diff: https://reviews.apache.org/r/53502/diff/
> 
> 
> Testing
> ---
> 
> Unit tests passing.
> 
> ...
> Tests run: 28, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 209.464 sec 
> - in org.apache.ambari.server.state.UpgradeHelperTest
> ...
> Tests run: 4727, Failures: 0, Errors: 0, Skipped: 42
> 
> 
> Thanks,
> 
> Duc Le
> 
>

Re: Review Request 53502: AMBARI-18681 Add host structured_out to manual task


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53502/#review155291
---




ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/HostOrderGrouping.java
 (line 179)


I did mark it as Ship It, but I'm curious if we want to have this as a 
separate enum?


- Jonathan Hurley


On Nov. 7, 2016, 10:42 p.m., Duc Le wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53502/
> ---
> 
> (Updated Nov. 7, 2016, 10:42 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, and Nate 
> Cole.
> 
> 
> Bugs: AMBARI-18681
> https://issues.apache.org/jira/browse/AMBARI-18681
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> AMBARI-18681 Add host structured_out to manual task
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/HostOrderGrouping.java
>  b999f2f 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/UpgradeHelperTest.java
>  78969e4 
> 
> Diff: https://reviews.apache.org/r/53502/diff/
> 
> 
> Testing
> ---
> 
> Unit tests passing.
> 
> ...
> Tests run: 28, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 209.464 sec 
> - in org.apache.ambari.server.state.UpgradeHelperTest
> ...
> Tests run: 4727, Failures: 0, Errors: 0, Skipped: 42
> 
> 
> Thanks,
> 
> Duc Le
> 
>

Re: Review Request 53502: AMBARI-18681 Add host structured_out to manual task


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53502/#review155290
---


Ship it!




Ship It!

- Jonathan Hurley


On Nov. 7, 2016, 10:42 p.m., Duc Le wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53502/
> ---
> 
> (Updated Nov. 7, 2016, 10:42 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, and Nate 
> Cole.
> 
> 
> Bugs: AMBARI-18681
> https://issues.apache.org/jira/browse/AMBARI-18681
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> AMBARI-18681 Add host structured_out to manual task
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/HostOrderGrouping.java
>  b999f2f 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/UpgradeHelperTest.java
>  78969e4 
> 
> Diff: https://reviews.apache.org/r/53502/diff/
> 
> 
> Testing
> ---
> 
> Unit tests passing.
> 
> ...
> Tests run: 28, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 209.464 sec 
> - in org.apache.ambari.server.state.UpgradeHelperTest
> ...
> Tests run: 4727, Failures: 0, Errors: 0, Skipped: 42
> 
> 
> Thanks,
> 
> Duc Le
> 
>

Re: Review Request 52369: AMBARI-12263: Support PAM as authentication mechanism for accessing Ambari UI/REST