Re: Review Request 59639: Blueprint export should allow tokenized values in SingleHostUpdater
> On June 5, 2017, 5:34 p.m., Alejandro Fernandez wrote: > > Ship It! Thank you Alejandro, could you please help push this to trunk? - Amruta --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59639/#review176927 --- On June 1, 2017, 9:58 p.m., Amruta Borkar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/59639/ > --- > > (Updated June 1, 2017, 9:58 p.m.) > > > Review request for Ambari, Attila Doroszlai, Di Li, and Robert Nettleton. > > > Bugs: AMBARI-21137 > https://issues.apache.org/jira/browse/AMBARI-21137 > > > Repository: ambari > > > Description > --- > > During upgrade Biginsights stack tokenizes value for > 'yarn.resourcemanager.webapp.https.address', therefore this property gets > removed by SingleHostUpdater during blueprint export. Next time when the > blueprint is used to create a cluster, this property is assigned default > value 'localhost' and exception is thrown. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java > 508bf1552d > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java > 24fc3c7435 > > > Diff: https://reviews.apache.org/r/59639/diff/2/ > > > Testing > --- > > Updated test cases to address the scenario. > > > Thanks, > > Amruta Borkar > >
Review Request 59639: Blueprint export should allow tokenized values in SingleHostUpdater
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59639/ --- Review request for Ambari, Di Li and Robert Nettleton. Bugs: AMBARI-21137 https://issues.apache.org/jira/browse/AMBARI-21137 Repository: ambari Description --- During upgrade Biginsights stack tokenizes value for 'yarn.resourcemanager.webapp.https.address', therefore this property gets removed by SingleHostUpdater during blueprint export. Next time when the blueprint is used to create a cluster, this property is assigned default value 'localhost' and exception is thrown. Diffs - ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java 508bf1552d ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java 24fc3c7435 Diff: https://reviews.apache.org/r/59639/diff/1/ Testing --- Updated test cases to address the scenario. Thanks, Amruta Borkar
Review Request 59609: Export cluster_name in the cluster template file
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59609/ --- Review request for Ambari, Di Li and Sangeeta Ravindran. Bugs: AMBARI-21132 https://issues.apache.org/jira/browse/AMBARI-21132 Repository: ambari Description --- Blueprint export from UI installer currently does not export cluster_name in the clustertemplate file. Adding patch to export the same Diffs - ambari-web/app/controllers/wizard/step8_controller.js f2b4ca77c9 Diff: https://reviews.apache.org/r/59609/diff/1/ Testing --- Tested manually, Attached clustertemplate.json file. File Attachments clustertemplate https://reviews.apache.org/media/uploaded/files/2017/05/26/59c6826a-e49a-4210-9079-b93cc0074c6f__clustertemplate_8.json Thanks, Amruta Borkar
Re: Review Request 59532: Add Confirmation pop up for 'Generate Blueprint' button
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59532/ --- (Updated May 24, 2017, 10:43 p.m.) Review request for Ambari, Alexandr Antonenko, Di Li, and Sangeeta Ravindran. Bugs: AMBARI-21104 https://issues.apache.org/jira/browse/AMBARI-21104 Repository: ambari Description --- Add Confirmation pop up for 'Generate Blueprint' button. Notify user that two files will be downloaded and provide information on how to use those files. Diffs - ambari-web/app/controllers/wizard/step8_controller.js f2b4ca77c9 ambari-web/app/messages.js 0c15a1935e ambari-web/app/templates/wizard/step8.hbs 907b3d4bbf Diff: https://reviews.apache.org/r/59532/diff/2/ Testing --- Tested manually. Attached image of the Confirmation box. File Attachments (updated) Confirmation pop up https://reviews.apache.org/media/uploaded/files/2017/05/24/b708ef1d-6a7f-4525-9dee-ecb11ab6d5fa__Snip20170524_9.png Thanks, Amruta Borkar
Re: Review Request 59532: Add Confirmation pop up for 'Generate Blueprint' button
> On May 24, 2017, 8:24 p.m., Di Li wrote: > > ambari-web/app/messages.js > > Lines 974 (patched) > > <https://reviews.apache.org/r/59532/diff/1/?file=1731664#file1731664line974> > > > > Can we update the msg to the following (I removed details on how bp > > register rest call), where is dynamically replaced by the cluster name > > the user typed in on the UI ? > > > > ... ... > > You will be downloading blueprint.json and clustertemplate.json to your > > local machine. > > > > You can use them to deploy a cluster via AMBARI REST API. > > Update clustertemplate.json with a custom cluster name if you do not > > want to use . > > > > If you want to use custom repo URLs, make sure to update them via > > Ambari REST API before deploying the cluster. Hello Di, Please review the updated patch. - Amruta --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59532/#review175987 --- On May 24, 2017, 10:17 p.m., Amruta Borkar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/59532/ > --- > > (Updated May 24, 2017, 10:17 p.m.) > > > Review request for Ambari, Alexandr Antonenko, Di Li, and Sangeeta Ravindran. > > > Bugs: AMBARI-21104 > https://issues.apache.org/jira/browse/AMBARI-21104 > > > Repository: ambari > > > Description > --- > > Add Confirmation pop up for 'Generate Blueprint' button. > Notify user that two files will be downloaded and provide information on how > to use those files. > > > Diffs > - > > ambari-web/app/controllers/wizard/step8_controller.js f2b4ca77c9 > ambari-web/app/messages.js 0c15a1935e > ambari-web/app/templates/wizard/step8.hbs 907b3d4bbf > > > Diff: https://reviews.apache.org/r/59532/diff/2/ > > > Testing > --- > > Tested manually. Attached image of the Confirmation box. > > > File Attachments > > > Generate_blueprint_confirmation > > https://reviews.apache.org/media/uploaded/files/2017/05/24/0c9086f1-b97b-4c2e-ac6b-d0666df7dfa0__Generate_blueprint_confirmation.png > > > Thanks, > > Amruta Borkar > >
Re: Review Request 59532: Add Confirmation pop up for 'Generate Blueprint' button
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59532/ --- (Updated May 24, 2017, 10:17 p.m.) Review request for Ambari, Alexandr Antonenko, Di Li, and Sangeeta Ravindran. Bugs: AMBARI-21104 https://issues.apache.org/jira/browse/AMBARI-21104 Repository: ambari Description --- Add Confirmation pop up for 'Generate Blueprint' button. Notify user that two files will be downloaded and provide information on how to use those files. Diffs (updated) - ambari-web/app/controllers/wizard/step8_controller.js f2b4ca77c9 ambari-web/app/messages.js 0c15a1935e ambari-web/app/templates/wizard/step8.hbs 907b3d4bbf Diff: https://reviews.apache.org/r/59532/diff/2/ Changes: https://reviews.apache.org/r/59532/diff/1-2/ Testing --- Tested manually. Attached image of the Confirmation box. File Attachments Generate_blueprint_confirmation https://reviews.apache.org/media/uploaded/files/2017/05/24/0c9086f1-b97b-4c2e-ac6b-d0666df7dfa0__Generate_blueprint_confirmation.png Thanks, Amruta Borkar
Review Request 59532: Add Confirmation pop up for 'Generate Blueprint' button
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59532/ --- Review request for Ambari, Alexandr Antonenko, Di Li, and Sangeeta Ravindran. Bugs: AMBARI-21104 https://issues.apache.org/jira/browse/AMBARI-21104 Repository: ambari Description --- Add Confirmation pop up for 'Generate Blueprint' button. Notify user that two files will be downloaded and provide information on how to use those files. Diffs - ambari-web/app/controllers/wizard/step8_controller.js f2b4ca77c9260ba123fdc165d680100f15641c22 ambari-web/app/messages.js 0c15a1935e5109689678e46eea01bfde532ce0e7 ambari-web/app/templates/wizard/step8.hbs 907b3d4bbfc859020f553b282452dd5f62cb9725 Diff: https://reviews.apache.org/r/59532/diff/1/ Testing --- Tested manually. Attached image of the Confirmation box. File Attachments Generate_blueprint_confirmation https://reviews.apache.org/media/uploaded/files/2017/05/24/0c9086f1-b97b-4c2e-ac6b-d0666df7dfa0__Generate_blueprint_confirmation.png Thanks, Amruta Borkar
Review Request 59497: Customize services shows inconsistent config-groups after clicking on Back
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59497/ --- Review request for Ambari, Aleksandr Kovalenko and Sangeeta Ravindran. Bugs: AMBARI-21094 https://issues.apache.org/jira/browse/AMBARI-21094 Repository: ambari Description --- Steps to reproduce: 1. Go to Customize Services screen 2. Create a Custom HostGroup 3. Click Back button 4. Click Next Button 5. See that Custom Hostgroup still exists, but all the data including assigned hosts is erased. Ambari typically erases all the changes when clicked on Back button, because the Custom HostGroup is not deleted but data related to it is deleted by Ambari, cluster state goes in inconsistent state. Diffs - ambari-web/app/controllers/wizard/step7_controller.js 6685c0189a Diff: https://reviews.apache.org/r/59497/diff/1/ Testing --- Tested manually, Attached output screenshots File Attachments HostGroupList_BeforeClickingBack https://reviews.apache.org/media/uploaded/files/2017/05/23/20029794-04d8-4611-83fe-884e30282ff7__HostGroupList_BeforeClickingBack.png BeforeCodeChange_ManageHostGroup_Customgroup_AfterClickingBack https://reviews.apache.org/media/uploaded/files/2017/05/23/3853d6c3-5fd0-4604-9923-602df062b51f__BeforeCodeChange_ManageHostGroup_Customgroup_AfterClickingBack.png BeforeCodeChange_ManageHostGroup_Defaultgroup_AfterClickingBack https://reviews.apache.org/media/uploaded/files/2017/05/23/2b28debe-b81f-4b25-946b-acdfc69a3042__BeforeCodeChange_ManageHostGroup_Defaultgroup_AfterClickingBack.png BeforeCodeChange_HostGroupList_AfterlickingBack https://reviews.apache.org/media/uploaded/files/2017/05/23/5c9805e0-41ae-42fa-a87d-393f0da8a144__BeforeCodeChange_HostGroupList_AfterlickingBack.png AfterCodeChange_HostGroupList https://reviews.apache.org/media/uploaded/files/2017/05/23/190f83cb-ce1b-4413-a9b3-cd988cd29133__AfterCodeChange_HostGroupList.png AfterCodeChange_ManageHostGroupPage https://reviews.apache.org/media/uploaded/files/2017/05/23/1bb710b5-f5c3-482b-8dca-dd3950ec2fbb__AfterCodeChange_ManageHostGroupPage.png Thanks, Amruta Borkar
Review Request 59497: Customize services shows inconsistent config-groups after clicking on Back
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59497/ --- Review request for Ambari, Aleksandr Kovalenko and Sangeeta Ravindran. Bugs: AMBARI-21094 https://issues.apache.org/jira/browse/AMBARI-21094 Repository: ambari Description --- Steps to reproduce: 1. Go to Customize Services screen 2. Create a Custom HostGroup 3. Click Back button 4. Click Next Button 5. See that Custom Hostgroup still exists, but all the data including assigned hosts is erased. Ambari typically erases all the changes when clicked on Back button, because the Custom HostGroup is not deleted but data related to it is deleted by Ambari, cluster state goes in inconsistent state. Diffs - ambari-web/app/controllers/wizard/step7_controller.js 6685c0189a Diff: https://reviews.apache.org/r/59497/diff/1/ Testing --- Tested manually, Attached output screenshots File Attachments HostGroupList_BeforeClickingBack https://reviews.apache.org/media/uploaded/files/2017/05/23/20029794-04d8-4611-83fe-884e30282ff7__HostGroupList_BeforeClickingBack.png BeforeCodeChange_ManageHostGroup_Customgroup_AfterClickingBack https://reviews.apache.org/media/uploaded/files/2017/05/23/3853d6c3-5fd0-4604-9923-602df062b51f__BeforeCodeChange_ManageHostGroup_Customgroup_AfterClickingBack.png BeforeCodeChange_ManageHostGroup_Defaultgroup_AfterClickingBack https://reviews.apache.org/media/uploaded/files/2017/05/23/2b28debe-b81f-4b25-946b-acdfc69a3042__BeforeCodeChange_ManageHostGroup_Defaultgroup_AfterClickingBack.png BeforeCodeChange_HostGroupList_AfterlickingBack https://reviews.apache.org/media/uploaded/files/2017/05/23/5c9805e0-41ae-42fa-a87d-393f0da8a144__BeforeCodeChange_HostGroupList_AfterlickingBack.png AfterCodeChange_HostGroupList https://reviews.apache.org/media/uploaded/files/2017/05/23/190f83cb-ce1b-4413-a9b3-cd988cd29133__AfterCodeChange_HostGroupList.png AfterCodeChange_ManageHostGroupPage https://reviews.apache.org/media/uploaded/files/2017/05/23/1bb710b5-f5c3-482b-8dca-dd3950ec2fbb__AfterCodeChange_ManageHostGroupPage.png Thanks, Amruta Borkar
Review Request 59497: Customize services shows inconsistent config-groups after clicking on Back
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59497/ --- Review request for Ambari, Aleksandr Kovalenko and Sangeeta Ravindran. Bugs: AMBARI-21094 https://issues.apache.org/jira/browse/AMBARI-21094 Repository: ambari Description --- Steps to reproduce: 1. Go to Customize Services screen 2. Create a Custom HostGroup 3. Click Back button 4. Click Next Button 5. See that Custom Hostgroup still exists, but all the data including assigned hosts is erased. Ambari typically erases all the changes when clicked on Back button, because the Custom HostGroup is not deleted but data related to it is deleted by Ambari, cluster state goes in inconsistent state. Diffs - ambari-web/app/controllers/wizard/step7_controller.js 6685c0189a Diff: https://reviews.apache.org/r/59497/diff/1/ Testing --- Tested manually, Attached output screenshots File Attachments HostGroupList_BeforeClickingBack https://reviews.apache.org/media/uploaded/files/2017/05/23/20029794-04d8-4611-83fe-884e30282ff7__HostGroupList_BeforeClickingBack.png BeforeCodeChange_ManageHostGroup_Customgroup_AfterClickingBack https://reviews.apache.org/media/uploaded/files/2017/05/23/3853d6c3-5fd0-4604-9923-602df062b51f__BeforeCodeChange_ManageHostGroup_Customgroup_AfterClickingBack.png BeforeCodeChange_ManageHostGroup_Defaultgroup_AfterClickingBack https://reviews.apache.org/media/uploaded/files/2017/05/23/2b28debe-b81f-4b25-946b-acdfc69a3042__BeforeCodeChange_ManageHostGroup_Defaultgroup_AfterClickingBack.png BeforeCodeChange_HostGroupList_AfterlickingBack https://reviews.apache.org/media/uploaded/files/2017/05/23/5c9805e0-41ae-42fa-a87d-393f0da8a144__BeforeCodeChange_HostGroupList_AfterlickingBack.png AfterCodeChange_HostGroupList https://reviews.apache.org/media/uploaded/files/2017/05/23/190f83cb-ce1b-4413-a9b3-cd988cd29133__AfterCodeChange_HostGroupList.png AfterCodeChange_ManageHostGroupPage https://reviews.apache.org/media/uploaded/files/2017/05/23/1bb710b5-f5c3-482b-8dca-dd3950ec2fbb__AfterCodeChange_ManageHostGroupPage.png Thanks, Amruta Borkar
Re: Review Request 59373: Address config-groups while exporting blueprint from UI
> On May 19, 2017, 8:28 a.m., Alexandr Antonenko wrote: > > Ship It! Hello Alexandr, Could you please help me push this to trunk? - Amruta --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59373/#review175476 --- On May 19, 2017, 5:01 p.m., Amruta Borkar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/59373/ > --- > > (Updated May 19, 2017, 5:01 p.m.) > > > Review request for Ambari, Alexandr Antonenko, Di Li, Robert Nettleton, and > Sangeeta Ravindran. > > > Bugs: AMBARI-21050 > https://issues.apache.org/jira/browse/AMBARI-21050 > > > Repository: ambari > > > Description > --- > > Exported blueprint should include config-group changes, which should be > exported as host-group level configuration. New host-group will be created > for host having unique config-group. > > > Diffs > - > > ambari-web/app/controllers/wizard/step8_controller.js 7e318e0e62 > ambari-web/test/controllers/wizard/step8_test.js 1a3214e263 > > > Diff: https://reviews.apache.org/r/59373/diff/2/ > > > Testing > --- > > Tested manually. Updated test cases. > > > File Attachments > > > exported blueprint > > https://reviews.apache.org/media/uploaded/files/2017/05/18/a01c8336-2e84-4c73-af54-c7ba60e7acb3__blueprint_expUI.json > exported cluster template > > https://reviews.apache.org/media/uploaded/files/2017/05/18/ee8f7e71-fea9-42ba-bcd1-9afc1a8654d6__clustertemplate_expUI.json > > > Thanks, > > Amruta Borkar > >
Re: Review Request 59373: Address config-groups while exporting blueprint from UI
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59373/ --- (Updated May 19, 2017, 5:01 p.m.) Review request for Ambari, Alexandr Antonenko, Di Li, Robert Nettleton, and Sangeeta Ravindran. Bugs: AMBARI-21050 https://issues.apache.org/jira/browse/AMBARI-21050 Repository: ambari Description --- Exported blueprint should include config-group changes, which should be exported as host-group level configuration. New host-group will be created for host having unique config-group. Diffs (updated) - ambari-web/app/controllers/wizard/step8_controller.js 7e318e0e62 ambari-web/test/controllers/wizard/step8_test.js 1a3214e263 Diff: https://reviews.apache.org/r/59373/diff/2/ Changes: https://reviews.apache.org/r/59373/diff/1-2/ Testing --- Tested manually. Updated test cases. File Attachments exported blueprint https://reviews.apache.org/media/uploaded/files/2017/05/18/a01c8336-2e84-4c73-af54-c7ba60e7acb3__blueprint_expUI.json exported cluster template https://reviews.apache.org/media/uploaded/files/2017/05/18/ee8f7e71-fea9-42ba-bcd1-9afc1a8654d6__clustertemplate_expUI.json Thanks, Amruta Borkar
Review Request 59373: Address config-groups while exporting blueprint from UI
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59373/ --- Review request for Ambari, Alexandr Antonenko, Di Li, Robert Nettleton, and Sangeeta Ravindran. Bugs: AMBARI-21050 https://issues.apache.org/jira/browse/AMBARI-21050 Repository: ambari Description --- Exported blueprint should include config-group changes, which should be exported as host-group level configuration. New host-group will be created for host having unique config-group. Diffs - ambari-web/app/controllers/wizard/step8_controller.js 7e318e0e62 ambari-web/test/controllers/wizard/step8_test.js 1a3214e263 Diff: https://reviews.apache.org/r/59373/diff/1/ Testing --- Tested manually. Updated test cases. File Attachments exported blueprint https://reviews.apache.org/media/uploaded/files/2017/05/18/a01c8336-2e84-4c73-af54-c7ba60e7acb3__blueprint_expUI.json exported cluster template https://reviews.apache.org/media/uploaded/files/2017/05/18/ee8f7e71-fea9-42ba-bcd1-9afc1a8654d6__clustertemplate_expUI.json Thanks, Amruta Borkar
Re: Review Request 58864: AMBARI-20698 Ability to export blueprint via Ambari installer UI
> On May 16, 2017, 3:14 p.m., Alexandr Antonenko wrote: > > Ship It! Thank you Alexandr. Could you please help me push this to trunk? - Amruta --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58864/#review175111 --- On May 11, 2017, 7:02 p.m., Amruta Borkar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/58864/ > --- > > (Updated May 11, 2017, 7:02 p.m.) > > > Review request for Ambari, Alexandr Antonenko, Di Li, Jaimin Jetly, Jayush > Luniya, Robert Nettleton, Sangeeta Ravindran, Sumit Mohanty, and Yusaku Sako. > > > Bugs: AMBARI-20698 > https://issues.apache.org/jira/browse/AMBARI-20698 > > > Repository: ambari > > > Description > --- > > UI be able to export a blueprint when a user reaches the Deploy page and the > bp can be used to install a cluster later on. > This patch exports blueprint and template. Currently HostGroups are formed > only based on the component mapping. In this patch configGroups are not > addressed. ConfigGroup and other improvements will be addressed in next > revisions of this functionality. > > > Diffs > - > > ambari-web/app/controllers/wizard/step8_controller.js 4678d03a48 > ambari-web/app/messages.js 8f8d981af7 > ambari-web/app/templates/wizard/step8.hbs ac32710468 > ambari-web/test/controllers/wizard/step8_test.js 7cdb69ab34 > > > Diff: https://reviews.apache.org/r/58864/diff/3/ > > > Testing > --- > > Manual Testing done. I have attached blueprint.json and clustertemplate.json > files that are currently generated with this patch applied. > > > File Attachments > > > exported_blueprint > > https://reviews.apache.org/media/uploaded/files/2017/04/28/897160b1-46d8-49d1-bc87-f14249718c26__exported_blueprint_.json > exported_clustertemplate > > https://reviews.apache.org/media/uploaded/files/2017/04/28/e53d037e-a77e-487e-ac08-ce754e4afc97__exported_clustertemplate.json > > > Thanks, > > Amruta Borkar > >
Re: Review Request 58864: AMBARI-20698 Ability to export blueprint via Ambari installer UI
> On May 11, 2017, 11:41 p.m., Alexandr Antonenko wrote:
> > ambari-web/app/controllers/wizard/step8_controller.js
> > Line 29 (original), 30 (patched)
> > <https://reviews.apache.org/r/58864/diff/3/?file=1710936#file1710936line30>
> >
> > is it aimed to hit all off the wizards that we are currently using ?
> >
> >
> > like
> >
> > /**
> >* @type {boolean}
> >*/
> > isAddService: Em.computed.equal('content.controllerName',
> > 'addServiceController'),
> >
> > /**
> >* @type {boolean}
> >*/
> > isAddHost: Em.computed.equal('content.controllerName',
> > 'addHostController'),
> >
> > /**
> >* @type {boolean}
> >*/
> > isInstaller: Em.computed.equal('content.controllerName',
> > 'installerController'),
> >
> >
> > because I didn't saw any chekings regaring this in your code
Hello Alexandr,
The 'Generate Blueprint' button should only show up on the last page in the
cluster installation wizard. I am checking it with
{{#unless App.router.clusterInstallCompleted}} in step8.hbs file.
This way the button will not show up on addService wizard.
- Amruta
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58864/#review174746
---
On May 11, 2017, 7:02 p.m., Amruta Borkar wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58864/
> ---
>
> (Updated May 11, 2017, 7:02 p.m.)
>
>
> Review request for Ambari, Alexandr Antonenko, Di Li, Jaimin Jetly, Jayush
> Luniya, Robert Nettleton, Sangeeta Ravindran, Sumit Mohanty, and Yusaku Sako.
>
>
> Bugs: AMBARI-20698
> https://issues.apache.org/jira/browse/AMBARI-20698
>
>
> Repository: ambari
>
>
> Description
> ---
>
> UI be able to export a blueprint when a user reaches the Deploy page and the
> bp can be used to install a cluster later on.
> This patch exports blueprint and template. Currently HostGroups are formed
> only based on the component mapping. In this patch configGroups are not
> addressed. ConfigGroup and other improvements will be addressed in next
> revisions of this functionality.
>
>
> Diffs
> -
>
> ambari-web/app/controllers/wizard/step8_controller.js 4678d03a48
> ambari-web/app/messages.js 8f8d981af7
> ambari-web/app/templates/wizard/step8.hbs ac32710468
> ambari-web/test/controllers/wizard/step8_test.js 7cdb69ab34
>
>
> Diff: https://reviews.apache.org/r/58864/diff/3/
>
>
> Testing
> ---
>
> Manual Testing done. I have attached blueprint.json and clustertemplate.json
> files that are currently generated with this patch applied.
>
>
> File Attachments
> ----
>
> exported_blueprint
>
> https://reviews.apache.org/media/uploaded/files/2017/04/28/897160b1-46d8-49d1-bc87-f14249718c26__exported_blueprint_.json
> exported_clustertemplate
>
> https://reviews.apache.org/media/uploaded/files/2017/04/28/e53d037e-a77e-487e-ac08-ce754e4afc97__exported_clustertemplate.json
>
>
> Thanks,
>
> Amruta Borkar
>
>
Re: Review Request 58864: AMBARI-20698 Ability to export blueprint via Ambari installer UI
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58864/ --- (Updated May 8, 2017, 9:45 p.m.) Review request for Ambari, Di Li, Jaimin Jetly, Jayush Luniya, Robert Nettleton, Sangeeta Ravindran, Sumit Mohanty, and Yusaku Sako. Changes --- Updated based on Sangeeta's review. Hello Jaimin, Yusaku, could you please review the patch? Thank you. Bugs: AMBARI-20698 https://issues.apache.org/jira/browse/AMBARI-20698 Repository: ambari Description --- UI be able to export a blueprint when a user reaches the Deploy page and the bp can be used to install a cluster later on. This patch exports blueprint and template. Currently HostGroups are formed only based on the component mapping. In this patch configGroups are not addressed. ConfigGroup and other improvements will be addressed in next revisions of this functionality. Diffs (updated) - ambari-web/app/controllers/wizard/step8_controller.js 4678d03a48 ambari-web/app/messages.js 8f8d981af7 ambari-web/app/templates/wizard/step8.hbs ac32710468 ambari-web/test/controllers/wizard/step8_test.js 7cdb69ab34 Diff: https://reviews.apache.org/r/58864/diff/3/ Changes: https://reviews.apache.org/r/58864/diff/2-3/ Testing --- Manual Testing done. I have attached blueprint.json and clustertemplate.json files that are currently generated with this patch applied. File Attachments exported_blueprint https://reviews.apache.org/media/uploaded/files/2017/04/28/897160b1-46d8-49d1-bc87-f14249718c26__exported_blueprint_.json exported_clustertemplate https://reviews.apache.org/media/uploaded/files/2017/04/28/e53d037e-a77e-487e-ac08-ce754e4afc97__exported_clustertemplate.json Thanks, Amruta Borkar
Re: Review Request 58864: AMBARI-20698 Ability to export blueprint via Ambari installer UI
> On May 2, 2017, 6:22 p.m., Robert Nettleton wrote: > > ambari-web/app/controllers/wizard/step8_controller.js > > Lines 1925 (patched) > > <https://reviews.apache.org/r/58864/diff/1/?file=1703676#file1703676line1925> > > > > Similar to my last point, the default provision_action is always > > "INSTALL_AND_START", so this might not need to be set in the Blueprint. Hello Robert, Thank you for your input, I think exporting these properties would make users aware of the options that are available to them. May be we can export both properties with default values? "config_recommendation_strategy" : "NEVER_APPLY" and "provision_action" : "INSTALL_AND_START"? @Di could you please provide your input? - Amruta --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58864/#review173615 --- On May 1, 2017, 9:06 p.m., Amruta Borkar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/58864/ > --- > > (Updated May 1, 2017, 9:06 p.m.) > > > Review request for Ambari, Di Li, Jaimin Jetly, Jayush Luniya, Robert > Nettleton, Sangeeta Ravindran, Sumit Mohanty, and Yusaku Sako. > > > Bugs: AMBARI-20698 > https://issues.apache.org/jira/browse/AMBARI-20698 > > > Repository: ambari > > > Description > --- > > UI be able to export a blueprint when a user reaches the Deploy page and the > bp can be used to install a cluster later on. > This patch exports blueprint and template. Currently HostGroups are formed > only based on the component mapping. In this patch configGroups are not > addressed. ConfigGroup and other improvements will be addressed in next > revisions of this functionality. > > > Diffs > - > > ambari-web/app/controllers/wizard/step8_controller.js 4678d03a48 > ambari-web/app/messages.js 8f8d981af7 > ambari-web/app/templates/wizard/step8.hbs ac32710468 > > > Diff: https://reviews.apache.org/r/58864/diff/1/ > > > Testing > --- > > Manual Testing done. I have attached blueprint.json and clustertemplate.json > files that are currently generated with this patch applied. > > > File Attachments > > > exported_blueprint > > https://reviews.apache.org/media/uploaded/files/2017/04/28/897160b1-46d8-49d1-bc87-f14249718c26__exported_blueprint_.json > exported_clustertemplate > > https://reviews.apache.org/media/uploaded/files/2017/04/28/e53d037e-a77e-487e-ac08-ce754e4afc97__exported_clustertemplate.json > > > Thanks, > > Amruta Borkar > >
Re: Review Request 57610: Filter out kerberos rules in exported blueprint
> On May 2, 2017, 4:32 p.m., Sandor Magyari wrote: > > Ship It! Thank you Sandor, could you please help me push this to trunk? - Amruta --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/57610/#review173600 --- On April 30, 2017, 4:03 a.m., Amruta Borkar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/57610/ > --- > > (Updated April 30, 2017, 4:03 a.m.) > > > Review request for Ambari, Di Li, Robert Levas, Robert Nettleton, and Sandor > Magyari. > > > Bugs: AMBARI-20366 > https://issues.apache.org/jira/browse/AMBARI-20366 > > > Repository: ambari > > > Description > --- > > If blueprint is exported from a kerberos enabled cluster Kerberos rules > export principal names which contain cluster name and Realm, this patch > filter out kerberos rules properties so that hardcoded cluster name and realm > are not exported. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java > 7381387b53 > > ambari-server/src/test/java/org/apache/ambari/server/api/query/render/ClusterBlueprintRendererTest.java > 13db5f8b56 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java > 8ff70a1d46 > > > Diff: https://reviews.apache.org/r/57610/diff/5/ > > > Testing > --- > > Tested manually. > Modified test cases. > > > Thanks, > > Amruta Borkar > >
Re: Review Request 57610: Filter out kerberos rules in exported blueprint
> On April 20, 2017, 4 p.m., Sandor Magyari wrote: > > I've suggested using the thread local but overall looking at the patch now, > > it seems to me a litle bit overcomplicated. > > May be it would much more simplier if we could pass authToLocal properties > > to KerberosAuthToLocalRulesFilter directly to the constructor. So instead > > of creating a static list of filters, we could create this list of filters > > (exportPropertyFilters) in doFilterPriorToExport method before filtering > > code - as it's not used anywhere else - where you can easily get > > authToLocal properties for the given cluster then pass it to > > KerberosAuthToLocalRulesFilter instance through the constructor. So I think > > actually we don't need to keep the list of filters statically all the time, > > we can create the list just for the time of export filtering. Hello Sandor, Could you please review the updated patch? - Amruta --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/57610/#review172506 ----------- On April 30, 2017, 4:03 a.m., Amruta Borkar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/57610/ > --- > > (Updated April 30, 2017, 4:03 a.m.) > > > Review request for Ambari, Di Li, Robert Levas, Robert Nettleton, and Sandor > Magyari. > > > Bugs: AMBARI-20366 > https://issues.apache.org/jira/browse/AMBARI-20366 > > > Repository: ambari > > > Description > --- > > If blueprint is exported from a kerberos enabled cluster Kerberos rules > export principal names which contain cluster name and Realm, this patch > filter out kerberos rules properties so that hardcoded cluster name and realm > are not exported. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java > 7381387b53 > > ambari-server/src/test/java/org/apache/ambari/server/api/query/render/ClusterBlueprintRendererTest.java > 13db5f8b56 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java > 8ff70a1d46 > > > Diff: https://reviews.apache.org/r/57610/diff/5/ > > > Testing > --- > > Tested manually. > Modified test cases. > > > Thanks, > > Amruta Borkar > >
Re: Review Request 57610: Filter out kerberos rules in exported blueprint
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/57610/ --- (Updated April 30, 2017, 4:03 a.m.) Review request for Ambari, Di Li, Robert Levas, Robert Nettleton, and Sandor Magyari. Bugs: AMBARI-20366 https://issues.apache.org/jira/browse/AMBARI-20366 Repository: ambari Description --- If blueprint is exported from a kerberos enabled cluster Kerberos rules export principal names which contain cluster name and Realm, this patch filter out kerberos rules properties so that hardcoded cluster name and realm are not exported. Diffs (updated) - ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java 7381387b53 ambari-server/src/test/java/org/apache/ambari/server/api/query/render/ClusterBlueprintRendererTest.java 13db5f8b56 ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java 8ff70a1d46 Diff: https://reviews.apache.org/r/57610/diff/5/ Changes: https://reviews.apache.org/r/57610/diff/4-5/ Testing --- Tested manually. Modified test cases. Thanks, Amruta Borkar
Review Request 58864: AMBARI-20698 Ability to export blueprint via Ambari installer UI
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58864/ --- Review request for Ambari, Di Li, Jayush Luniya, Robert Nettleton, and Sumit Mohanty. Bugs: AMBARI-20698 https://issues.apache.org/jira/browse/AMBARI-20698 Repository: ambari Description --- UI be able to export a blueprint when a user reaches the Deploy page and the bp can be used to install a cluster later on. This patch exports blueprint and template. Currently HostGroups are formed only based on the component mapping. In this patch configGroups are not addressed. ConfigGroup and other improvements will be addressed in next revisions of this functionality. Diffs - ambari-web/app/controllers/wizard/step8_controller.js 4678d03a48 ambari-web/app/messages.js 8f8d981af7 ambari-web/app/templates/wizard/step8.hbs ac32710468 Diff: https://reviews.apache.org/r/58864/diff/1/ Testing --- Manual Testing done. I have attached blueprint.json and clustertemplate.json files that are currently generated with this patch applied. File Attachments exported_blueprint https://reviews.apache.org/media/uploaded/files/2017/04/28/897160b1-46d8-49d1-bc87-f14249718c26__exported_blueprint_.json exported_clustertemplate https://reviews.apache.org/media/uploaded/files/2017/04/28/e53d037e-a77e-487e-ac08-ce754e4afc97__exported_clustertemplate.json Thanks, Amruta Borkar
Re: Review Request 57610: Tokenize kerberos principal name appearing in kerberos rules in exported blueprint
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/57610/ --- (Updated April 18, 2017, 9:13 p.m.) Review request for Ambari, Di Li, Robert Levas, Robert Nettleton, and Sandor Magyari. Changes --- Updated patch for latest trunk code Bugs: AMBARI-20366 https://issues.apache.org/jira/browse/AMBARI-20366 Repository: ambari Description --- If blueprint is exported from a kerberos enabled cluster Kerberos rules export principal names which contain cluster name and Realm, this exports existing cluster name and realm name as tokens and replaces those tokens with new values of cluster name and realm during successive cluster deployments. Diffs (updated) - ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java bb771a54a6 ambari-server/src/test/java/org/apache/ambari/server/api/query/render/ClusterBlueprintRendererTest.java 13db5f8b56 ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java 5c1836a87d Diff: https://reviews.apache.org/r/57610/diff/4/ Changes: https://reviews.apache.org/r/57610/diff/3-4/ Testing --- Tested manually. Modified test cases. Thanks, Amruta Borkar
Re: Review Request 57610: Tokenize kerberos principal name appearing in kerberos rules in exported blueprint
> On April 10, 2017, 8:54 p.m., Sandor Magyari wrote: > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java > > Lines 3042 (patched) > > <https://reviews.apache.org/r/57610/diff/2/?file=1685430#file1685430line3042> > > > > Caching these properties is a good idea, but I think we should be able > > to handle multi clusters, so keep theese properties separatly for each > > cluster and ideally these should be cached only for the time of export, > > since next time the state of the cluster may change. We don't have such > > filters yet, may be we can cache theese props in a HashMap by clusterId, in > > ThreadLocal and an init method to PropertyFilter where we can reset this > > map prior beginning the export. Hello Sandor, I modififed the patch to address this problem, could you please review it? - Amruta --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/57610/#review171474 ------- On April 13, 2017, 6:09 p.m., Amruta Borkar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/57610/ > --- > > (Updated April 13, 2017, 6:09 p.m.) > > > Review request for Ambari, Di Li, Robert Levas, Robert Nettleton, and Sandor > Magyari. > > > Bugs: AMBARI-20366 > https://issues.apache.org/jira/browse/AMBARI-20366 > > > Repository: ambari > > > Description > --- > > If blueprint is exported from a kerberos enabled cluster Kerberos rules > export principal names which contain cluster name and Realm, this exports > existing cluster name and realm name as tokens and replaces those tokens with > new values of cluster name and realm during successive cluster deployments. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java > bb771a54a6 > > ambari-server/src/test/java/org/apache/ambari/server/api/query/render/ClusterBlueprintRendererTest.java > 13db5f8b56 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java > dba40437ae > > > Diff: https://reviews.apache.org/r/57610/diff/3/ > > > Testing > --- > > Tested manually. > Modified test cases. > > > Thanks, > > Amruta Borkar > >
Re: Review Request 57610: Tokenize kerberos principal name appearing in kerberos rules in exported blueprint
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/57610/ --- (Updated April 13, 2017, 6:09 p.m.) Review request for Ambari, Di Li, Robert Levas, Robert Nettleton, and Sandor Magyari. Bugs: AMBARI-20366 https://issues.apache.org/jira/browse/AMBARI-20366 Repository: ambari Description --- If blueprint is exported from a kerberos enabled cluster Kerberos rules export principal names which contain cluster name and Realm, this exports existing cluster name and realm name as tokens and replaces those tokens with new values of cluster name and realm during successive cluster deployments. Diffs (updated) - ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java bb771a54a6 ambari-server/src/test/java/org/apache/ambari/server/api/query/render/ClusterBlueprintRendererTest.java 13db5f8b56 ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java dba40437ae Diff: https://reviews.apache.org/r/57610/diff/3/ Changes: https://reviews.apache.org/r/57610/diff/2-3/ Testing --- Tested manually. Modified test cases. Thanks, Amruta Borkar
Re: Review Request 58122: Blueprint export fails if config-type is not mapped to any service after upgrade
> On April 6, 2017, 11:13 p.m., Alejandro Fernandez wrote: > > Ship It! Thank you Alejandro, Could you please help me push this to trunk? - Amruta --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58122/#review171274 --- On April 5, 2017, 11:45 p.m., Amruta Borkar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/58122/ > --- > > (Updated April 5, 2017, 11:45 p.m.) > > > Review request for Ambari, Alejandro Fernandez, Di Li, and Robert Nettleton. > > > Bugs: AMBARI-20551 > https://issues.apache.org/jira/browse/AMBARI-20551 > > > Repository: ambari > > > Description > --- > > Following error is thrown while exporting a blueprint from cluster. > ERROR [ambari-client-thread-8726] ReadHandler:99 - Bad request: > java.lang.IllegalArgumentException: Specified configuration type is not > associated with any service: storm-site > at > org.apache.ambari.server.controller.internal.Stack.getServiceForConfigType(Stack.java:494) > at > org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessor$StackPropertyTypeFilter.isPropertyIncluded(BlueprintConfigurationProcessor.java:2946) > at > org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessor.shouldPropertyBeExcludedForBlueprintExport(BlueprintConfigurationProcessor.java:939) > at > org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessor.doFilterPriorToExport(BlueprintConfigurationProcessor.java:439) > at > org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessor.doUpdateForBlueprintExport(BlueprintConfigurationProcessor.java:416) > at > org.apache.ambari.server.api.query.render.ClusterBlueprintRenderer.createBlueprintResource(ClusterBlueprintRenderer.java:186) > at > org.apache.ambari.server.api.query.render.ClusterBlueprintRenderer.finalizeResult(ClusterBlueprintRenderer.java:141) > at org.apache.ambari.server.api.query.QueryImpl.getResult(QueryImpl.java:839) > at org.apache.ambari.server.api.query.QueryImpl.execute(QueryImpl.java:223) > at > org.apache.ambari.server.api.handlers.ReadHandler.handleRequest(ReadHandler.java:77) > at > org.apache.ambari.server.api.services.BaseRequest.process(BaseRequest.java:145) > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java > db1aa074d4 > > > Diff: https://reviews.apache.org/r/58122/diff/4/ > > > Testing > --- > > Tested manually. Log shows warning about unmapped config-type but blueprint > gets exported. > > > Thanks, > > Amruta Borkar > >
Re: Review Request 58122: Blueprint export fails if config-type is not mapped to any service after upgrade
> On April 5, 2017, 5:32 p.m., Alejandro Fernandez wrote: > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java > > Lines 525 (patched) > > <https://reviews.apache.org/r/58122/diff/3/?file=1685433#file1685433line525> > > > > Missing spaces after the periods Thanks Alejandro, Could you please help me ppush this to trunk? - Amruta --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58122/#review171149 --- On April 5, 2017, 11:45 p.m., Amruta Borkar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/58122/ > --- > > (Updated April 5, 2017, 11:45 p.m.) > > > Review request for Ambari, Alejandro Fernandez, Di Li, and Robert Nettleton. > > > Bugs: AMBARI-20551 > https://issues.apache.org/jira/browse/AMBARI-20551 > > > Repository: ambari > > > Description > --- > > Following error is thrown while exporting a blueprint from cluster. > ERROR [ambari-client-thread-8726] ReadHandler:99 - Bad request: > java.lang.IllegalArgumentException: Specified configuration type is not > associated with any service: storm-site > at > org.apache.ambari.server.controller.internal.Stack.getServiceForConfigType(Stack.java:494) > at > org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessor$StackPropertyTypeFilter.isPropertyIncluded(BlueprintConfigurationProcessor.java:2946) > at > org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessor.shouldPropertyBeExcludedForBlueprintExport(BlueprintConfigurationProcessor.java:939) > at > org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessor.doFilterPriorToExport(BlueprintConfigurationProcessor.java:439) > at > org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessor.doUpdateForBlueprintExport(BlueprintConfigurationProcessor.java:416) > at > org.apache.ambari.server.api.query.render.ClusterBlueprintRenderer.createBlueprintResource(ClusterBlueprintRenderer.java:186) > at > org.apache.ambari.server.api.query.render.ClusterBlueprintRenderer.finalizeResult(ClusterBlueprintRenderer.java:141) > at org.apache.ambari.server.api.query.QueryImpl.getResult(QueryImpl.java:839) > at org.apache.ambari.server.api.query.QueryImpl.execute(QueryImpl.java:223) > at > org.apache.ambari.server.api.handlers.ReadHandler.handleRequest(ReadHandler.java:77) > at > org.apache.ambari.server.api.services.BaseRequest.process(BaseRequest.java:145) > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java > db1aa074d4 > > > Diff: https://reviews.apache.org/r/58122/diff/4/ > > > Testing > --- > > Tested manually. Log shows warning about unmapped config-type but blueprint > gets exported. > > > Thanks, > > Amruta Borkar > >
Re: Review Request 58122: Blueprint export fails if config-type is not mapped to any service after upgrade
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58122/ --- (Updated April 5, 2017, 11:45 p.m.) Review request for Ambari, Alejandro Fernandez, Di Li, and Robert Nettleton. Changes --- Modified to address the feedback comments. Bugs: AMBARI-20551 https://issues.apache.org/jira/browse/AMBARI-20551 Repository: ambari Description --- Following error is thrown while exporting a blueprint from cluster. ERROR [ambari-client-thread-8726] ReadHandler:99 - Bad request: java.lang.IllegalArgumentException: Specified configuration type is not associated with any service: storm-site at org.apache.ambari.server.controller.internal.Stack.getServiceForConfigType(Stack.java:494) at org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessor$StackPropertyTypeFilter.isPropertyIncluded(BlueprintConfigurationProcessor.java:2946) at org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessor.shouldPropertyBeExcludedForBlueprintExport(BlueprintConfigurationProcessor.java:939) at org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessor.doFilterPriorToExport(BlueprintConfigurationProcessor.java:439) at org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessor.doUpdateForBlueprintExport(BlueprintConfigurationProcessor.java:416) at org.apache.ambari.server.api.query.render.ClusterBlueprintRenderer.createBlueprintResource(ClusterBlueprintRenderer.java:186) at org.apache.ambari.server.api.query.render.ClusterBlueprintRenderer.finalizeResult(ClusterBlueprintRenderer.java:141) at org.apache.ambari.server.api.query.QueryImpl.getResult(QueryImpl.java:839) at org.apache.ambari.server.api.query.QueryImpl.execute(QueryImpl.java:223) at org.apache.ambari.server.api.handlers.ReadHandler.handleRequest(ReadHandler.java:77) at org.apache.ambari.server.api.services.BaseRequest.process(BaseRequest.java:145) Diffs (updated) - ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java db1aa074d4 Diff: https://reviews.apache.org/r/58122/diff/4/ Changes: https://reviews.apache.org/r/58122/diff/3-4/ Testing --- Tested manually. Log shows warning about unmapped config-type but blueprint gets exported. Thanks, Amruta Borkar
Re: Review Request 58122: Blueprint export fails if config-type is not mapped to any service after upgrade
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58122/ --- (Updated April 5, 2017, 4:58 p.m.) Review request for Ambari, Di Li and Robert Nettleton. Changes --- Updated patch to address the comments Bugs: AMBARI-20551 https://issues.apache.org/jira/browse/AMBARI-20551 Repository: ambari Description --- Following error is thrown while exporting a blueprint from cluster. ERROR [ambari-client-thread-8726] ReadHandler:99 - Bad request: java.lang.IllegalArgumentException: Specified configuration type is not associated with any service: storm-site at org.apache.ambari.server.controller.internal.Stack.getServiceForConfigType(Stack.java:494) at org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessor$StackPropertyTypeFilter.isPropertyIncluded(BlueprintConfigurationProcessor.java:2946) at org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessor.shouldPropertyBeExcludedForBlueprintExport(BlueprintConfigurationProcessor.java:939) at org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessor.doFilterPriorToExport(BlueprintConfigurationProcessor.java:439) at org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessor.doUpdateForBlueprintExport(BlueprintConfigurationProcessor.java:416) at org.apache.ambari.server.api.query.render.ClusterBlueprintRenderer.createBlueprintResource(ClusterBlueprintRenderer.java:186) at org.apache.ambari.server.api.query.render.ClusterBlueprintRenderer.finalizeResult(ClusterBlueprintRenderer.java:141) at org.apache.ambari.server.api.query.QueryImpl.getResult(QueryImpl.java:839) at org.apache.ambari.server.api.query.QueryImpl.execute(QueryImpl.java:223) at org.apache.ambari.server.api.handlers.ReadHandler.handleRequest(ReadHandler.java:77) at org.apache.ambari.server.api.services.BaseRequest.process(BaseRequest.java:145) Diffs (updated) - ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java db1aa074d4 Diff: https://reviews.apache.org/r/58122/diff/3/ Changes: https://reviews.apache.org/r/58122/diff/2-3/ Testing --- Tested manually. Log shows warning about unmapped config-type but blueprint gets exported. Thanks, Amruta Borkar
Re: Review Request 57610: Tokenize kerberos principal name appearing in kerberos rules in exported blueprint
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/57610/ --- (Updated April 5, 2017, 4:56 p.m.) Review request for Ambari, Di Li, Robert Levas, Robert Nettleton, and Sandor Magyari. Changes --- Updated patch to filter out auth_to_local properties. Bugs: AMBARI-20366 https://issues.apache.org/jira/browse/AMBARI-20366 Repository: ambari Description --- If blueprint is exported from a kerberos enabled cluster Kerberos rules export principal names which contain cluster name and Realm, this exports existing cluster name and realm name as tokens and replaces those tokens with new values of cluster name and realm during successive cluster deployments. Diffs (updated) - ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java db1aa074d4 ambari-server/src/test/java/org/apache/ambari/server/api/query/render/ClusterBlueprintRendererTest.java 13db5f8b56 ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java dba40437ae Diff: https://reviews.apache.org/r/57610/diff/2/ Changes: https://reviews.apache.org/r/57610/diff/1-2/ Testing --- Tested manually. Modified test cases. Thanks, Amruta Borkar
Re: Review Request 57563: Kerberos principal creation fails during blueprint install when kdc_hosts is not specified in blueprint
> On April 3, 2017, 6:11 p.m., Robert Nettleton wrote: > > Thanks for providing this patch, but I don't really think that this is the > > correct way to solve this problem. > > > > The Blueprint export shouldn't really modify any configuration in the > > cluster. The export process may filter out some information (passwords, > > hostnames for un-managed services, etc), but no configuration property > > changes should be made. > > > > In the future, it would be great if Blueprints provided a validation step > > for the configuration in a Blueprint/Cluster Creation template, as that > > would resolve the initial issue described. Unfortunately, such a framework > > does not yet exist in Blueprints, and would require refactoring of the > > stack definitions, such that the Blueprint processor could validate > > configuration properties in a generic fashion. Hello Robert, Thanks for the feedback. Would it be a good idea to handle the validation mentioned by Robert Levas in BlueprintValidatorImpl ? - Amruta --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/57563/#review170908 ------- On April 3, 2017, 4:42 p.m., Amruta Borkar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/57563/ > --- > > (Updated April 3, 2017, 4:42 p.m.) > > > Review request for Ambari, Di Li and Robert Levas. > > > Bugs: AMBARI-20386 > https://issues.apache.org/jira/browse/AMBARI-20386 > > > Repository: ambari > > > Description > --- > > Blueprint validation does not prompt if kdc_hosts is not specified in the > blueprint. After service installation, deployment fails during Create > Principal stage. As /etc/krb5.conf is overwritten with blank values. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java > db1aa074d4 > > ambari-server/src/test/java/org/apache/ambari/server/api/query/render/ClusterBlueprintRendererTest.java > 13db5f8b56 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java > dba40437ae > > > Diff: https://reviews.apache.org/r/57563/diff/2/ > > > Testing > --- > > Manual testing done, if no kdc_hosts is mentioned then validation exception > will be thrown during blueprint registration. > > > Thanks, > > Amruta Borkar > >
Re: Review Request 57563: Kerberos principal creation fails during blueprint install when kdc_hosts is not specified in blueprint
> On April 3, 2017, 4:49 p.m., Di Li wrote: > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java > > Lines 931 (patched) > > <https://reviews.apache.org/r/57563/diff/2/?file=1683692#file1683692line931> > > > > Does this mean bp export will always set it to false ? yes, it will always be exported as false. > On April 3, 2017, 4:49 p.m., Di Li wrote: > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java > > Line 21 (original), 21 (patched) > > <https://reviews.apache.org/r/57563/diff/2/?file=1683694#file1683694line21> > > > > why change the junit classes imported ? junit.framework.Assert.assertEquals; is deprecated, I saw it is replaced by 'org.junit.Assert.assertEquals;' in another class, therefore I made same changes in this class. - Amruta --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/57563/#review170888 ------- On April 3, 2017, 4:42 p.m., Amruta Borkar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/57563/ > --- > > (Updated April 3, 2017, 4:42 p.m.) > > > Review request for Ambari, Di Li and Robert Levas. > > > Bugs: AMBARI-20386 > https://issues.apache.org/jira/browse/AMBARI-20386 > > > Repository: ambari > > > Description > --- > > Blueprint validation does not prompt if kdc_hosts is not specified in the > blueprint. After service installation, deployment fails during Create > Principal stage. As /etc/krb5.conf is overwritten with blank values. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java > db1aa074d4 > > ambari-server/src/test/java/org/apache/ambari/server/api/query/render/ClusterBlueprintRendererTest.java > 13db5f8b56 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java > dba40437ae > > > Diff: https://reviews.apache.org/r/57563/diff/2/ > > > Testing > --- > > Manual testing done, if no kdc_hosts is mentioned then validation exception > will be thrown during blueprint registration. > > > Thanks, > > Amruta Borkar > >
Re: Review Request 57563: Kerberos principal creation fails during blueprint install when kdc_hosts is not specified in blueprint
> On March 14, 2017, 11:01 a.m., Robert Levas wrote: > > ambari-server/src/main/resources/common-services/KERBEROS/1.10.3-10/configuration/kerberos-env.xml > > Line 135 (original), 135 (patched) > > <https://reviews.apache.org/r/57563/diff/1/?file=1662766#file1662766line135> > > > > We cannot force `kerberos-env/kdc-hosts` to require input. This value > > is only required when when the `kerberos-env/kdc-type` is "mit-kdc", "ipa", > > or "active-directory". If `kerberos-env/kdc-type` is "none", which > > indicates that the user will manually set up the Kerbeos infrastructure, > > then `kerberos-env/kdc-hosts` is not needed. > > > > Also, no mater what the value of `kerberos-env/kdc-type` is, if the > > user opts to manaully manage the krb5.conf file > > (`krb5-conf/manage_krb5_conf` is set to "false"), then > > `kerberos-env/kdc-hosts` is not required. Hello Robert, Thank you for the information. In case of blueprint deployment, the property krb5-conf/manage_krb5_conf gets exported as true (default value). Therefore, if the same blueprint is used to deploy cluster then Ambari overrides the /etc/krb5.conf file with black values, as 'kdc_hosts' is not present in the blueprint. I modified the patch to export krb5-conf/manage_krb5_conf as false, so that using same blueprint would not run into error condition. Could you provide your input? Thanks. - Amruta --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/57563/#review168881 --- On April 3, 2017, 4:42 p.m., Amruta Borkar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/57563/ > --- > > (Updated April 3, 2017, 4:42 p.m.) > > > Review request for Ambari, Di Li and Robert Levas. > > > Bugs: AMBARI-20386 > https://issues.apache.org/jira/browse/AMBARI-20386 > > > Repository: ambari > > > Description > --- > > Blueprint validation does not prompt if kdc_hosts is not specified in the > blueprint. After service installation, deployment fails during Create > Principal stage. As /etc/krb5.conf is overwritten with blank values. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java > db1aa074d4 > > ambari-server/src/test/java/org/apache/ambari/server/api/query/render/ClusterBlueprintRendererTest.java > 13db5f8b56 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java > dba40437ae > > > Diff: https://reviews.apache.org/r/57563/diff/2/ > > > Testing > --- > > Manual testing done, if no kdc_hosts is mentioned then validation exception > will be thrown during blueprint registration. > > > Thanks, > > Amruta Borkar > >
Re: Review Request 57563: Kerberos principal creation fails during blueprint install when kdc_hosts is not specified in blueprint
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/57563/ --- (Updated April 3, 2017, 4:42 p.m.) Review request for Ambari, Di Li and Robert Levas. Bugs: AMBARI-20386 https://issues.apache.org/jira/browse/AMBARI-20386 Repository: ambari Description --- Blueprint validation does not prompt if kdc_hosts is not specified in the blueprint. After service installation, deployment fails during Create Principal stage. As /etc/krb5.conf is overwritten with blank values. Diffs (updated) - ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java db1aa074d4 ambari-server/src/test/java/org/apache/ambari/server/api/query/render/ClusterBlueprintRendererTest.java 13db5f8b56 ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java dba40437ae Diff: https://reviews.apache.org/r/57563/diff/2/ Changes: https://reviews.apache.org/r/57563/diff/1-2/ Testing --- Manual testing done, if no kdc_hosts is mentioned then validation exception will be thrown during blueprint registration. Thanks, Amruta Borkar
Review Request 58122: Blueprint export fails if config-type is not mapped to any service after upgrade
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58122/ --- Review request for Ambari, Di Li and Robert Nettleton. Bugs: AMBARI-20551 https://issues.apache.org/jira/browse/AMBARI-20551 Repository: ambari Description --- Following error is thrown while exporting a blueprint from cluster. ERROR [ambari-client-thread-8726] ReadHandler:99 - Bad request: java.lang.IllegalArgumentException: Specified configuration type is not associated with any service: storm-site at org.apache.ambari.server.controller.internal.Stack.getServiceForConfigType(Stack.java:494) at org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessor$StackPropertyTypeFilter.isPropertyIncluded(BlueprintConfigurationProcessor.java:2946) at org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessor.shouldPropertyBeExcludedForBlueprintExport(BlueprintConfigurationProcessor.java:939) at org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessor.doFilterPriorToExport(BlueprintConfigurationProcessor.java:439) at org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessor.doUpdateForBlueprintExport(BlueprintConfigurationProcessor.java:416) at org.apache.ambari.server.api.query.render.ClusterBlueprintRenderer.createBlueprintResource(ClusterBlueprintRenderer.java:186) at org.apache.ambari.server.api.query.render.ClusterBlueprintRenderer.finalizeResult(ClusterBlueprintRenderer.java:141) at org.apache.ambari.server.api.query.QueryImpl.getResult(QueryImpl.java:839) at org.apache.ambari.server.api.query.QueryImpl.execute(QueryImpl.java:223) at org.apache.ambari.server.api.handlers.ReadHandler.handleRequest(ReadHandler.java:77) at org.apache.ambari.server.api.services.BaseRequest.process(BaseRequest.java:145) Diffs - ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java 25f2001 Diff: https://reviews.apache.org/r/58122/diff/1/ Testing --- Tested manually. Log shows warning about unmapped config-type but blueprint gets exported. Thanks, Amruta Borkar
Re: Review Request 57610: Tokenize kerberos principal name appearing in kerberos rules in exported blueprint
> On March 16, 2017, 1:12 p.m., Sandor Magyari wrote: > > The question is here whether is useful or not to export auth_to_local > > properties, since they are generated at deploy time anyway. May be would be > > better to exclude from export. > > Could you please also add Robert Levas as a reviewer? > > Robert Levas wrote: > I think in general, the auth-to-local rules should not be exported as > part of the blueprint, since as Sandor mentions that they are dynamically > created anyways. The content of the rules are cluster specific. So if the > realm changes, the rules must change as well. Same with the cluster name - if > the cluster name is used as the value to help create unique principal names. > > That said, +1 for Sandor's comment of removing auth-to-local rules from > the exported BP. Hello Sandor, Robert I am trying to tokenize rather that filtering out from blueprint, to preserve any custom mapping rules user may have added to auth_to_local. Could you please give suggestion for this case? Thank you - Amruta --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/57610/#review169142 ----------- On March 14, 2017, 5:21 p.m., Amruta Borkar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/57610/ > --- > > (Updated March 14, 2017, 5:21 p.m.) > > > Review request for Ambari, Di Li, Robert Nettleton, and Sandor Magyari. > > > Bugs: AMBARI-20366 > https://issues.apache.org/jira/browse/AMBARI-20366 > > > Repository: ambari > > > Description > --- > > If blueprint is exported from a kerberos enabled cluster Kerberos rules > export principal names which contain cluster name and Realm, this exports > existing cluster name and realm name as tokens and replaces those tokens with > new values of cluster name and realm during successive cluster deployments. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/api/query/render/ClusterBlueprintRenderer.java > 5e19a6c > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java > 5732a1c > > ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterConfigurationRequest.java > e29417b > > ambari-server/src/test/java/org/apache/ambari/server/api/query/render/ClusterBlueprintRendererTest.java > 75ffd31 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java > d160050 > > ambari-server/src/test/java/org/apache/ambari/server/topology/ClusterConfigurationRequestTest.java > c97c568 > > > Diff: https://reviews.apache.org/r/57610/diff/1/ > > > Testing > --- > > Tested manually. > Modified test cases. > > > Thanks, > > Amruta Borkar > >
Review Request 57610: Tokenize kerberos principal name appearing in kerberos rules in exported blueprint
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/57610/ --- Review request for Ambari, Di Li, Robert Nettleton, and Sandor Magyari. Bugs: AMBARI-20366 https://issues.apache.org/jira/browse/AMBARI-20366 Repository: ambari Description --- If blueprint is exported from a kerberos enabled cluster Kerberos rules export principal names which contain cluster name and Realm, this exports existing cluster name and realm name as tokens and replaces those tokens with new values of cluster name and realm during successive cluster deployments. Diffs - ambari-server/src/main/java/org/apache/ambari/server/api/query/render/ClusterBlueprintRenderer.java 5e19a6c ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java 5732a1c ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterConfigurationRequest.java e29417b ambari-server/src/test/java/org/apache/ambari/server/api/query/render/ClusterBlueprintRendererTest.java 75ffd31 ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java d160050 ambari-server/src/test/java/org/apache/ambari/server/topology/ClusterConfigurationRequestTest.java c97c568 Diff: https://reviews.apache.org/r/57610/diff/1/ Testing --- Tested manually. Modified test cases. Thanks, Amruta Borkar
Review Request 57563: Kerberos principal creation fails during blueprint install when kdc_hosts is not specified in blueprint
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/57563/ --- Review request for Ambari, Di Li and Robert Levas. Bugs: AMBARI-20386 https://issues.apache.org/jira/browse/AMBARI-20386 Repository: ambari Description --- Blueprint validation does not prompt if kdc_hosts is not specified in the blueprint. After service installation, deployment fails during Create Principal stage. As /etc/krb5.conf is overwritten with blank values. Diffs - ambari-server/src/main/resources/common-services/KERBEROS/1.10.3-10/configuration/kerberos-env.xml 965ee13 Diff: https://reviews.apache.org/r/57563/diff/1/ Testing --- Manual testing done, if no kdc_hosts is mentioned then validation exception will be thrown during blueprint registration. Thanks, Amruta Borkar
Re: Review Request 57297: Cluster deployment using blueprint with empty configuration doesn't work with stack advisor enabled
> On March 8, 2017, 10:49 p.m., Alejandro Fernandez wrote: > > Ship It! Thank you Alejandro, Could you please help me push this to trunk? - Amruta --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/57297/#review168366 --- On March 8, 2017, 11:08 p.m., Amruta Borkar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/57297/ > --- > > (Updated March 8, 2017, 11:08 p.m.) > > > Review request for Ambari, Alejandro Fernandez, Di Li, Robert Nettleton, and > Sumit Mohanty. > > > Bugs: AMBARI-20288 > https://issues.apache.org/jira/browse/AMBARI-20288 > > > Repository: ambari > > > Description > --- > > When a blueprint deployment is made with empty configuration for a particular > config-type and when stack advisor is returning a ValueAttributeInfo for for > that config-type with a property marked "true" for "Delete" flag > > Cluster deployment gets stuck at TOPOLOGY_RESOLUTION phase showing message > Config type not resolved yet, Blueprint deployment will wait until > configuration update is completed. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java > d4880b9 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java > 928d1e0 > > > Diff: https://reviews.apache.org/r/57297/diff/1/ > > > Testing > --- > > Manual testing done. > Modified test case to address the scenario. > > > Thanks, > > Amruta Borkar > >
Review Request 57060: HostCleaup.py throws exception and exits when hostcheck_custom_actions.result not present
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/57060/ --- Review request for Ambari, Di Li and Jonathan Hurley. Bugs: AMBARI-20192 https://issues.apache.org/jira/browse/AMBARI-20192 Repository: ambari Description --- HostCleanup script throws following exception and exits when hostcheck_custom_actions.result not present [root@tsar1 ~]# /usr/lib/python2.6/site-packages/ambari_agent/HostCleanup.py You have elected to remove all users as well. If it is not intended then use option --skip "users". Do you want to continue [y/n] (y)y Host Check results not found. There is no /var/lib/ambari-agent/data/hostcheck_custom_actions.result. Do you want to run host checks [y/n] (y)n Traceback (most recent call last): File "/usr/lib/python2.6/site-packages/ambari_agent/HostCleanup.py", line 705, in main() File "/usr/lib/python2.6/site-packages/ambari_agent/HostCleanup.py", line 689, in main with open(hostcheckfileca, "rb") as f2: IOError: [Errno 2] No such file or directory: '/var/lib/ambari-agent/data/hostcheck_custom_actions.result' Diffs - ambari-agent/src/main/python/ambari_agent/HostCleanup.py 7ee3b1c Diff: https://reviews.apache.org/r/57060/diff/ Testing --- Manually tested. Cleanup completed successfully when no hostcheck_custom_actions.result file was present Thanks, Amruta Borkar
Re: Review Request 56627: Cluster deployment fails if cluster template has fewer host_groups than that in blueprint
> On Feb. 21, 2017, 5:41 p.m., Sandor Magyari wrote: > > Ship It! Thank you Sandor, I don't have commit rights on trunk. Could you please help me push this to trunk? - Amruta --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/56627/#review166193 --- On Feb. 20, 2017, 7:04 p.m., Amruta Borkar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/56627/ > --- > > (Updated Feb. 20, 2017, 7:04 p.m.) > > > Review request for Ambari, Di Li, Robert Nettleton, and Sandor Magyari. > > > Bugs: AMBARI-19991 > https://issues.apache.org/jira/browse/AMBARI-19991 > > > Repository: ambari > > > Description > --- > > Cluster creation REST API request gets accepted when, cluster template does > not list all the host_groups mentioned in the blueprint. But cluster > deployment fails showing error message in the logs > Caused by: java.lang.IllegalArgumentException: > TopologyManager.ConfigureClusterTask - prerequisites for config request > processing not yet satisfied > at > org.apache.ambari.server.topology.TopologyManager$ConfigureClusterTask.call(TopologyManager.java:908) > at > org.apache.ambari.server.topology.TopologyManager$ConfigureClusterTask.call(TopologyManager.java:889) > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/topology/TopologyManager.java > c4c3dd2 > > Diff: https://reviews.apache.org/r/56627/diff/ > > > Testing > --- > > Tested manually. > > > Thanks, > > Amruta Borkar > >
Re: Review Request 56627: Cluster deployment fails if cluster template has fewer host_groups than that in blueprint
> On Feb. 16, 2017, 12:05 p.m., Sandor Magyari wrote: > > Yes Robert is right, so there's no such validation because in general is > > not necessary to have all hosts from all hostgroups. For ex. let's say you > > have a 'worker' hostgroup which contains only Slave or Client components > > and you want to add hosts to this hostgroup later. In this case cluster > > deployment is started. > > If you check ConfigureClusterTask.getTopologyRequiredHostGroups() basically > > those are the hostgroups that are required to start configurations and > > these are typically hostgroups containing Master components. > > So if want to validate then we should validate these host groups are > > in Cluster template. The problem is that you can retrive this from > > ClusterConfigurationRequest.getRequiredHostGroups() which is created in > > TopologyManager.provisionCluster() (line:304) so at a later point then > > validateTopology, when cluster resources are already created. So in order > > to retrieve required HostGroups we need to create a lot of things like > > Cluster/components in db/memory which we need to revert all in case of > > throwing an exception from here TopologyManager.provisionCluster() > > (line:304). > > Another approach would be to check required HostGroups at the end of > > TopologyManager.provisionCluster() and in case of missing required hgroups > > we can return a message about this problem, which user can correct by > > sending a scale request specifying host(s) for missing hgroups. Hello Sandor, Thank you for providing this information. As cluster deployment request is getting persisted before getRequiredHostGroups() method call. I am not able to return a message, instead I addded error message to address this special case where a master host group is not mentioned in cluster template file. Could you please review the change? - Amruta --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/56627/#review165833 --- On Feb. 15, 2017, 7:30 p.m., Amruta Borkar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/56627/ > --- > > (Updated Feb. 15, 2017, 7:30 p.m.) > > > Review request for Ambari, Di Li, Robert Nettleton, and Sandor Magyari. > > > Bugs: AMBARI-19991 > https://issues.apache.org/jira/browse/AMBARI-19991 > > > Repository: ambari > > > Description > --- > > Cluster creation REST API request gets accepted when, cluster template does > not list all the host_groups mentioned in the blueprint. But cluster > deployment fails showing error message in the logs > Caused by: java.lang.IllegalArgumentException: > TopologyManager.ConfigureClusterTask - prerequisites for config request > processing not yet satisfied > at > org.apache.ambari.server.topology.TopologyManager$ConfigureClusterTask.call(TopologyManager.java:908) > at > org.apache.ambari.server.topology.TopologyManager$ConfigureClusterTask.call(TopologyManager.java:889) > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterTopologyImpl.java > d0a4df9 > > ambari-server/src/test/java/org/apache/ambari/server/topology/ClusterTopologyImplTest.java > 1f31d05 > > Diff: https://reviews.apache.org/r/56627/diff/ > > > Testing > --- > > Tested manually, updated test cases > > > Thanks, > > Amruta Borkar > >
Review Request 56627: Cluster deployment fails if cluster template has fewer host_groups than that in blueprint
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/56627/ --- Review request for Ambari, Di Li and Robert Nettleton. Bugs: AMBARI-19991 https://issues.apache.org/jira/browse/AMBARI-19991 Repository: ambari Description --- Cluster creation REST API request gets accepted when, cluster template does not list all the host_groups mentioned in the blueprint. But cluster deployment fails showing error message in the logs Caused by: java.lang.IllegalArgumentException: TopologyManager.ConfigureClusterTask - prerequisites for config request processing not yet satisfied at org.apache.ambari.server.topology.TopologyManager$ConfigureClusterTask.call(TopologyManager.java:908) at org.apache.ambari.server.topology.TopologyManager$ConfigureClusterTask.call(TopologyManager.java:889) Diffs - ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterTopologyImpl.java d0a4df9 ambari-server/src/test/java/org/apache/ambari/server/topology/ClusterTopologyImplTest.java 1f31d05 Diff: https://reviews.apache.org/r/56627/diff/ Testing --- Tested manually, updated test cases Thanks, Amruta Borkar
Re: Review Request 56235: findLatestServiceConfigsByStack query returns deleted config group
> On Feb. 3, 2017, 1:28 p.m., Di Li wrote: > > Ship It! > > Di Li wrote: > pushed to trunk as > https://git-wip-us.apache.org/repos/asf?p=ambari.git;a=commit;h=ecfc951b5c8589b9225684c3ad2660cdb4ef4445 Thank you Di. - Amruta --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/56235/#review164118 --- On Feb. 2, 2017, 8:01 p.m., Amruta Borkar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/56235/ > --- > > (Updated Feb. 2, 2017, 8:01 p.m.) > > > Review request for Ambari, Di Li and Nate Cole. > > > Bugs: AMBARI-19813 > https://issues.apache.org/jira/browse/AMBARI-19813 > > > Repository: ambari > > > Description > --- > > Query returns deleted config groups as latest active config group. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/orm/entities/ServiceConfigEntity.java > 8a1b316 > > ambari-server/src/test/java/org/apache/ambari/server/orm/dao/ServiceConfigDAOTest.java > 5890c35 > > Diff: https://reviews.apache.org/r/56235/diff/ > > > Testing > --- > > Modified test cases. > > > Thanks, > > Amruta Borkar > >
Review Request 56235: findLatestServiceConfigsByStack query returns deleted config group
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/56235/ --- Review request for Ambari, Di Li and Nate Cole. Bugs: AMBARI-19813 https://issues.apache.org/jira/browse/AMBARI-19813 Repository: ambari Description --- Query returns deleted config groups as latest active config group. Diffs - ambari-server/src/main/java/org/apache/ambari/server/orm/entities/ServiceConfigEntity.java 8a1b316 ambari-server/src/test/java/org/apache/ambari/server/orm/dao/ServiceConfigDAOTest.java 5890c35 Diff: https://reviews.apache.org/r/56235/diff/ Testing --- Modified test cases. Thanks, Amruta Borkar
Re: Review Request 55935: When hiveserver2 LDAP authentication is enabled hive.server2.authentication.ldap.url property gets exported with hardcoded hostname
> On Jan. 26, 2017, 4:25 p.m., Robert Nettleton wrote: > > Ship It! Hello Robert, Could you help me push this to trunk? Thank you. - Amruta --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/55935/#review163133 --- On Jan. 25, 2017, 8:58 p.m., Amruta Borkar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/55935/ > --- > > (Updated Jan. 25, 2017, 8:58 p.m.) > > > Review request for Ambari, Di Li and Robert Nettleton. > > > Bugs: AMBARI-19700 > https://issues.apache.org/jira/browse/AMBARI-19700 > > > Repository: ambari > > > Description > --- > > hardcoded hostname should not get exported. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java > 25d6145 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java > 0ad0351 > > Diff: https://reviews.apache.org/r/55935/diff/ > > > Testing > --- > > Modified test cases and tested manually to check exported blueprint for > hostname withn and outside of the cluster. > > > Thanks, > > Amruta Borkar > >
Review Request 55935: When hiveserver2 LDAP authentication is enabled hive.server2.authentication.ldap.url property gets exported with hardcoded hostname
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/55935/ --- Review request for Ambari, Di Li and Robert Nettleton. Bugs: AMBARI-19700 https://issues.apache.org/jira/browse/AMBARI-19700 Repository: ambari Description --- hardcoded hostname should not get exported. Diffs - ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java 25d6145 ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java 0ad0351 Diff: https://reviews.apache.org/r/55935/diff/ Testing --- Modified test cases and tested manually to check exported blueprint for hostname withn and outside of the cluster. Thanks, Amruta Borkar
Re: Review Request 55789: Deleted configuration groups are displayed along with active group with is_current value set to true using Ambari API's
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/55789/ --- (Updated Jan. 25, 2017, 6:25 p.m.) Review request for Ambari, Di Li, Jaimin Jetly, and Nate Cole. Bugs: AMBARI-19641 https://issues.apache.org/jira/browse/AMBARI-19641 Repository: ambari Description (updated) --- Following REST API returns deleted config group along with active groups with is_current value set to true. http://hostname/api/v1/clusters/kafkatest68/configurations/service_config_versions?service_name=HDFS_current=true Please refer to the attachments ConfigHistory.png and service_config_versions_HDFS_is_current as an example. Diffs - ambari-server/src/main/java/org/apache/ambari/server/orm/entities/ServiceConfigEntity.java 2d6ca20 ambari-server/src/test/java/org/apache/ambari/server/orm/dao/ServiceConfigDAOTest.java aafe557 Diff: https://reviews.apache.org/r/55789/diff/ Testing --- Tested manually. After applying patch, deleted config group entries were not returned for the above REST API call. Thanks, Amruta Borkar
Re: Review Request 55789: Deleted configuration groups are displayed along with active group with is_current value set to true using Ambari API's
> On Jan. 23, 2017, 5:40 p.m., Nate Cole wrote: > > There should be a test (and/or a negative test) covering this change. Hello Nate, I have added test cases to cover the patch. Could you please review? Thanks. - Amruta --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/55789/#review162659 --- On Jan. 24, 2017, 4:50 p.m., Amruta Borkar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/55789/ > --- > > (Updated Jan. 24, 2017, 4:50 p.m.) > > > Review request for Ambari, Di Li, Jaimin Jetly, and Nate Cole. > > > Bugs: AMBARI-19641 > https://issues.apache.org/jira/browse/AMBARI-19641 > > > Repository: ambari > > > Description > --- > > Following REST API returns deleted config group along with active groups with > is_current value set to true. > > http://hostname/api/v1/clusters/kafkatest68/configurations/service_config_versions?service_name=HDFS_current=true > > Please here to the attachments ConfigHistory.png and > service_config_versions_HDFS_is_current as an example. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/orm/entities/ServiceConfigEntity.java > 2d6ca20 > > ambari-server/src/test/java/org/apache/ambari/server/orm/dao/ServiceConfigDAOTest.java > aafe557 > > Diff: https://reviews.apache.org/r/55789/diff/ > > > Testing > --- > > Tested manually. After applying patch, deleted config group entries were not > returned for the above REST API call. > > > Thanks, > > Amruta Borkar > >
Re: Review Request 55789: Deleted configuration groups are displayed along with active group with is_current value set to true using Ambari API's
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/55789/ --- (Updated Jan. 24, 2017, 4:50 p.m.) Review request for Ambari, Di Li, Jaimin Jetly, and Nate Cole. Changes --- Added test cases Bugs: AMBARI-19641 https://issues.apache.org/jira/browse/AMBARI-19641 Repository: ambari Description --- Following REST API returns deleted config group along with active groups with is_current value set to true. http://hostname/api/v1/clusters/kafkatest68/configurations/service_config_versions?service_name=HDFS_current=true Please here to the attachments ConfigHistory.png and service_config_versions_HDFS_is_current as an example. Diffs (updated) - ambari-server/src/main/java/org/apache/ambari/server/orm/entities/ServiceConfigEntity.java 2d6ca20 ambari-server/src/test/java/org/apache/ambari/server/orm/dao/ServiceConfigDAOTest.java aafe557 Diff: https://reviews.apache.org/r/55789/diff/ Testing --- Tested manually. After applying patch, deleted config group entries were not returned for the above REST API call. Thanks, Amruta Borkar
Re: Review Request 55789: Deleted configuration groups are displayed along with active group with is_current value set to true using Ambari API's
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/55789/ --- (Updated Jan. 20, 2017, 11:49 p.m.) Review request for Ambari, Di Li and Jaimin Jetly. Bugs: AMBARI-19641 https://issues.apache.org/jira/browse/AMBARI-19641 Repository: ambari Description --- Following REST API returns deleted config group along with active groups with is_current value set to true. http://hostname/api/v1/clusters/kafkatest68/configurations/service_config_versions?service_name=HDFS_current=true Please here to the attachments ConfigHistory.png and service_config_versions_HDFS_is_current as an example. Diffs (updated) - ambari-server/src/main/java/org/apache/ambari/server/orm/entities/ServiceConfigEntity.java 2d6ca20 Diff: https://reviews.apache.org/r/55789/diff/ Testing --- Tested manually. After applying patch, deleted config group entries were not returned for the above REST API call. Thanks, Amruta Borkar
Review Request 55789: Deleted configuration groups are displayed along with active group with is_current value set to true using Ambari API's
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/55789/ --- Review request for Ambari, Di Li and Jaimin Jetly. Bugs: AMBARI-19641 https://issues.apache.org/jira/browse/AMBARI-19641 Repository: ambari Description --- Following REST API returns deleted config group along with active groups with is_current value set to true. http://hostname/api/v1/clusters/kafkatest68/configurations/service_config_versions?service_name=HDFS_current=true Please here to the attachments ConfigHistory.png and service_config_versions_HDFS_is_current as an example. Diffs - ambari-server/src/main/java/org/apache/ambari/server/orm/entities/ServiceConfigEntity.java 2d6ca20 Diff: https://reviews.apache.org/r/55789/diff/ Testing --- Tested manually. After applying patch, deleted config group entries were not returned for the above REST API call. Thanks, Amruta Borkar
Re: Review Request 53213: Exporting blueprint from kerberos enabled cluster, exports hardcoded values cluster name and realm in principal_name property
> On Jan. 19, 2017, 11:49 p.m., Alejandro Fernandez wrote: > > Ship It! Thank you Alejandro, I don't have commit rights on trunk. Could you please help me push this to trunk? - Amruta --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/53213/#review162365 --- On Jan. 19, 2017, 6:16 p.m., Amruta Borkar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/53213/ > --- > > (Updated Jan. 19, 2017, 6:16 p.m.) > > > Review request for Ambari, Di Li, Jaimin Jetly, Jayush Luniya, Robert Levas, > and Robert Nettleton. > > > Bugs: AMBARI-18692 > https://issues.apache.org/jira/browse/AMBARI-18692 > > > Repository: ambari > > > Description > --- > > Exporting blueprint from kerberos enabled cluster, exports hardcoded values > cluster name and realm in principal_name property. > When the same blueprint is used to create another cluster with different > name, service start fail with following error: > "resource_management.core.exceptions.Fail: Execution of '/usr/bin/kinit -kt > /etc/security/keytabs/hdfs.headless.keytab [keytab_name_in_blueprint] eturned > 1. kinit: Keytab contains no suitable keys for [keytab_name_in_blueprint] > while getting initial credentials" > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java > 737e8de > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/Stack.java > 16f75ee > > ambari-server/src/main/java/org/apache/ambari/server/state/PropertyInfo.java > 01c8a89 > > ambari-server/src/main/resources/common-services/ACCUMULO/1.6.1.2.2.0/configuration/accumulo-env.xml > 1d330dd > > ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/configuration/infra-solr-env.xml > d7ae236 > > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-hbase-security-site.xml > 2be101b > > ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/configuration/application-properties.xml > 9efa4f9 > > ambari-server/src/main/resources/common-services/DRUID/0.9.2/configuration/druid-common.xml > 0ded5d6 > > ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/configuration/hbase-env.xml > 2ab6aa0 > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/configuration/hadoop-env.xml > c2f37c1 > > ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/configuration/hadoop-env.xml > 24032fa > > ambari-server/src/main/resources/common-services/KAFKA/0.8.1/configuration/kafka-env.xml > e8a76b6 > > ambari-server/src/main/resources/common-services/KERBEROS/1.10.3-10/configuration/kerberos-env.xml > 7016437 > > ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/configuration/knox-env.xml > 2e5a026 > > ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logfeeder-env.xml > fb641a8 > > ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-env.xml > a6f1502 > > ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-properties.xml > 47872ee > > ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-admin-site.xml > 36ebc8c > > ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-tagsync-site.xml > a0bd322 > > ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-ugsync-site.xml > d267b75 > > ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/configuration/kms-site.xml > 1bce4e6 > > ambari-server/src/main/resources/common-services/SPARK/1.2.1/configuration/spark-defaults.xml > b85cf50 > > ambari-server/src/main/resources/common-services/SPARK/1.2.1/configuration/spark-env.xml > 71a42f7 > > ambari-server/src/main/resources/common-services/SPARK2/2.0.0/configuration/spark2-defaults.xml > a5115af > > ambari-server/src/main/resources/common-services/SPARK2/2.0.0/configuration/spark2-env.xml > b78fd79 > > ambari-server/src/main/resources/common-services/STORM/0.9.1/configuration/storm-env.xml > 5d0b2ff > > ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/configuratio
Re: Review Request 53213: Exporting blueprint from kerberos enabled cluster, exports hardcoded values cluster name and realm in principal_name property
> On Jan. 19, 2017, 7:24 p.m., Alejandro Fernandez wrote: > > ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5/configuration/zookeeper-env.xml, > > line 75 > > <https://reviews.apache.org/r/53213/diff/4/?file=1609336#file1609336line75> > > > > This is also needed in ZK 3.4.9 Hello Alejandro, I am not seeing any configuration xml file in ZK 3.4.9 . I only see metainfo.xml and stack_advisor.py . There fore don't see any Kerberos principal name property. Is there anything I am missing? - Amruta --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/53213/#review162335 --- On Jan. 19, 2017, 6:16 p.m., Amruta Borkar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/53213/ > --- > > (Updated Jan. 19, 2017, 6:16 p.m.) > > > Review request for Ambari, Di Li, Jaimin Jetly, Jayush Luniya, Robert Levas, > and Robert Nettleton. > > > Bugs: AMBARI-18692 > https://issues.apache.org/jira/browse/AMBARI-18692 > > > Repository: ambari > > > Description > --- > > Exporting blueprint from kerberos enabled cluster, exports hardcoded values > cluster name and realm in principal_name property. > When the same blueprint is used to create another cluster with different > name, service start fail with following error: > "resource_management.core.exceptions.Fail: Execution of '/usr/bin/kinit -kt > /etc/security/keytabs/hdfs.headless.keytab [keytab_name_in_blueprint] eturned > 1. kinit: Keytab contains no suitable keys for [keytab_name_in_blueprint] > while getting initial credentials" > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java > 737e8de > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/Stack.java > 16f75ee > > ambari-server/src/main/java/org/apache/ambari/server/state/PropertyInfo.java > 01c8a89 > > ambari-server/src/main/resources/common-services/ACCUMULO/1.6.1.2.2.0/configuration/accumulo-env.xml > 1d330dd > > ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/configuration/infra-solr-env.xml > d7ae236 > > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-hbase-security-site.xml > 2be101b > > ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/configuration/application-properties.xml > 9efa4f9 > > ambari-server/src/main/resources/common-services/DRUID/0.9.2/configuration/druid-common.xml > 0ded5d6 > > ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/configuration/hbase-env.xml > 2ab6aa0 > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/configuration/hadoop-env.xml > c2f37c1 > > ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/configuration/hadoop-env.xml > 24032fa > > ambari-server/src/main/resources/common-services/KAFKA/0.8.1/configuration/kafka-env.xml > e8a76b6 > > ambari-server/src/main/resources/common-services/KERBEROS/1.10.3-10/configuration/kerberos-env.xml > 7016437 > > ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/configuration/knox-env.xml > 2e5a026 > > ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logfeeder-env.xml > fb641a8 > > ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-env.xml > a6f1502 > > ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-properties.xml > 47872ee > > ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-admin-site.xml > 36ebc8c > > ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-tagsync-site.xml > a0bd322 > > ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-ugsync-site.xml > d267b75 > > ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/configuration/kms-site.xml > 1bce4e6 > > ambari-server/src/main/resources/common-services/SPARK/1.2.1/configuration/spark-defaults.xml > b85cf50 > > ambari-server/src/main/resources/common-services/SPARK/1.2.1/configuration/spark-env.xml > 71a42f7 > > ambari-server/src/main/resources/common-services/SPARK2/2.0.0/configurat
Re: Review Request 53213: Exporting blueprint from kerberos enabled cluster, exports hardcoded values cluster name and realm in principal_name property
> On Jan. 18, 2017, 7:05 p.m., Alejandro Fernandez wrote: > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/configuration/hadoop-env.xml, > > line 238 > > <https://reviews.apache.org/r/53213/diff/3/?file=1603263#file1603263line238> > > > > This annotation is missing in HDFS 3.0.0.3.0, YARN 3.0.0.3.0, and ZK > > 3.4.9 Hello Alejandro, Thank you for pointing that out. I missed HDFS 3.0.0.3.0. I could not find any Kerberos principal name property in YARN 3.0.0.3.0 and ZK 3.4.9. If you find everything ok, could you please help me push this to trunk? Thank you, - Amruta --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/53213/#review162157 ------- On Jan. 19, 2017, 6:16 p.m., Amruta Borkar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/53213/ > --- > > (Updated Jan. 19, 2017, 6:16 p.m.) > > > Review request for Ambari, Di Li, Jaimin Jetly, Jayush Luniya, Robert Levas, > and Robert Nettleton. > > > Bugs: AMBARI-18692 > https://issues.apache.org/jira/browse/AMBARI-18692 > > > Repository: ambari > > > Description > --- > > Exporting blueprint from kerberos enabled cluster, exports hardcoded values > cluster name and realm in principal_name property. > When the same blueprint is used to create another cluster with different > name, service start fail with following error: > "resource_management.core.exceptions.Fail: Execution of '/usr/bin/kinit -kt > /etc/security/keytabs/hdfs.headless.keytab [keytab_name_in_blueprint] eturned > 1. kinit: Keytab contains no suitable keys for [keytab_name_in_blueprint] > while getting initial credentials" > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java > 737e8de > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/Stack.java > 16f75ee > > ambari-server/src/main/java/org/apache/ambari/server/state/PropertyInfo.java > 01c8a89 > > ambari-server/src/main/resources/common-services/ACCUMULO/1.6.1.2.2.0/configuration/accumulo-env.xml > 1d330dd > > ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/configuration/infra-solr-env.xml > d7ae236 > > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-hbase-security-site.xml > 2be101b > > ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/configuration/application-properties.xml > 9efa4f9 > > ambari-server/src/main/resources/common-services/DRUID/0.9.2/configuration/druid-common.xml > 0ded5d6 > > ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/configuration/hbase-env.xml > 2ab6aa0 > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/configuration/hadoop-env.xml > c2f37c1 > > ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/configuration/hadoop-env.xml > 24032fa > > ambari-server/src/main/resources/common-services/KAFKA/0.8.1/configuration/kafka-env.xml > e8a76b6 > > ambari-server/src/main/resources/common-services/KERBEROS/1.10.3-10/configuration/kerberos-env.xml > 7016437 > > ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/configuration/knox-env.xml > 2e5a026 > > ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logfeeder-env.xml > fb641a8 > > ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-env.xml > a6f1502 > > ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-properties.xml > 47872ee > > ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-admin-site.xml > 36ebc8c > > ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-tagsync-site.xml > a0bd322 > > ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-ugsync-site.xml > d267b75 > > ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/configuration/kms-site.xml > 1bce4e6 > > ambari-server/src/main/resources/common-services/SPARK/1.2.1/configuration/spark-defaults.xml > b85cf50 > > ambari-server/src/main/resources/common-services/SPARK/1.2.1/configuration/spark-env.xml > 71a42f7 &g
Re: Review Request 53213: Exporting blueprint from kerberos enabled cluster, exports hardcoded values cluster name and realm in principal_name property
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/53213/ --- (Updated Jan. 19, 2017, 6:16 p.m.) Review request for Ambari, Di Li, Jaimin Jetly, Jayush Luniya, Robert Levas, and Robert Nettleton. Bugs: AMBARI-18692 https://issues.apache.org/jira/browse/AMBARI-18692 Repository: ambari Description --- Exporting blueprint from kerberos enabled cluster, exports hardcoded values cluster name and realm in principal_name property. When the same blueprint is used to create another cluster with different name, service start fail with following error: "resource_management.core.exceptions.Fail: Execution of '/usr/bin/kinit -kt /etc/security/keytabs/hdfs.headless.keytab [keytab_name_in_blueprint] eturned 1. kinit: Keytab contains no suitable keys for [keytab_name_in_blueprint] while getting initial credentials" Diffs (updated) - ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java 737e8de ambari-server/src/main/java/org/apache/ambari/server/controller/internal/Stack.java 16f75ee ambari-server/src/main/java/org/apache/ambari/server/state/PropertyInfo.java 01c8a89 ambari-server/src/main/resources/common-services/ACCUMULO/1.6.1.2.2.0/configuration/accumulo-env.xml 1d330dd ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/configuration/infra-solr-env.xml d7ae236 ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-hbase-security-site.xml 2be101b ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/configuration/application-properties.xml 9efa4f9 ambari-server/src/main/resources/common-services/DRUID/0.9.2/configuration/druid-common.xml 0ded5d6 ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/configuration/hbase-env.xml 2ab6aa0 ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/configuration/hadoop-env.xml c2f37c1 ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/configuration/hadoop-env.xml 24032fa ambari-server/src/main/resources/common-services/KAFKA/0.8.1/configuration/kafka-env.xml e8a76b6 ambari-server/src/main/resources/common-services/KERBEROS/1.10.3-10/configuration/kerberos-env.xml 7016437 ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/configuration/knox-env.xml 2e5a026 ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logfeeder-env.xml fb641a8 ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-env.xml a6f1502 ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-properties.xml 47872ee ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-admin-site.xml 36ebc8c ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-tagsync-site.xml a0bd322 ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-ugsync-site.xml d267b75 ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/configuration/kms-site.xml 1bce4e6 ambari-server/src/main/resources/common-services/SPARK/1.2.1/configuration/spark-defaults.xml b85cf50 ambari-server/src/main/resources/common-services/SPARK/1.2.1/configuration/spark-env.xml 71a42f7 ambari-server/src/main/resources/common-services/SPARK2/2.0.0/configuration/spark2-defaults.xml a5115af ambari-server/src/main/resources/common-services/SPARK2/2.0.0/configuration/spark2-env.xml b78fd79 ambari-server/src/main/resources/common-services/STORM/0.9.1/configuration/storm-env.xml 5d0b2ff ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/configuration/zeppelin-env.xml e6c59d5 ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5/configuration/zookeeper-env.xml 696b28a ambari-server/src/main/resources/configuration-schema.xsd 17f7393 Diff: https://reviews.apache.org/r/53213/diff/ Testing --- Tested manually. Suitable keytabs are generated automatically when not mentioned in blueprint. Service starts succeeded with a blueprint exported with the code change. Modified existing unit test cases. Thanks, Amruta Borkar
Re: Review Request 53213: Exporting blueprint from kerberos enabled cluster, exports hardcoded values cluster name and realm in principal_name property
> On Jan. 17, 2017, 2:15 p.m., Robert Levas wrote: > > Ship It! Thank you. - Amruta --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/53213/#review161866 --- On Jan. 12, 2017, 12:33 a.m., Amruta Borkar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/53213/ > --- > > (Updated Jan. 12, 2017, 12:33 a.m.) > > > Review request for Ambari, Di Li, Jaimin Jetly, Jayush Luniya, Robert Levas, > and Robert Nettleton. > > > Bugs: AMBARI-18692 > https://issues.apache.org/jira/browse/AMBARI-18692 > > > Repository: ambari > > > Description > --- > > Exporting blueprint from kerberos enabled cluster, exports hardcoded values > cluster name and realm in principal_name property. > When the same blueprint is used to create another cluster with different > name, service start fail with following error: > "resource_management.core.exceptions.Fail: Execution of '/usr/bin/kinit -kt > /etc/security/keytabs/hdfs.headless.keytab [keytab_name_in_blueprint] eturned > 1. kinit: Keytab contains no suitable keys for [keytab_name_in_blueprint] > while getting initial credentials" > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java > 737e8de > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/Stack.java > 16f75ee > > ambari-server/src/main/java/org/apache/ambari/server/state/PropertyInfo.java > 01c8a89 > > ambari-server/src/main/resources/common-services/ACCUMULO/1.6.1.2.2.0/configuration/accumulo-env.xml > 1d330dd > > ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/configuration/infra-solr-env.xml > d7ae236 > > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-hbase-security-site.xml > 2be101b > > ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/configuration/application-properties.xml > 9efa4f9 > > ambari-server/src/main/resources/common-services/DRUID/0.9.2/configuration/druid-common.xml > 0ded5d6 > > ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/configuration/hbase-env.xml > 2ab6aa0 > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/configuration/hadoop-env.xml > c2f37c1 > > ambari-server/src/main/resources/common-services/KAFKA/0.8.1/configuration/kafka-env.xml > e8a76b6 > > ambari-server/src/main/resources/common-services/KERBEROS/1.10.3-10/configuration/kerberos-env.xml > 7016437 > > ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/configuration/knox-env.xml > 2e5a026 > > ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logfeeder-env.xml > fb641a8 > > ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-env.xml > a6f1502 > > ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-properties.xml > 47872ee > > ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-admin-site.xml > 36ebc8c > > ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-tagsync-site.xml > a0bd322 > > ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-ugsync-site.xml > d267b75 > > ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/configuration/kms-site.xml > 1bce4e6 > > ambari-server/src/main/resources/common-services/SPARK/1.2.1/configuration/spark-defaults.xml > b85cf50 > > ambari-server/src/main/resources/common-services/SPARK/1.2.1/configuration/spark-env.xml > 71a42f7 > > ambari-server/src/main/resources/common-services/SPARK2/2.0.0/configuration/spark2-defaults.xml > a5115af > > ambari-server/src/main/resources/common-services/SPARK2/2.0.0/configuration/spark2-env.xml > b78fd79 > > ambari-server/src/main/resources/common-services/STORM/0.9.1/configuration/storm-env.xml > 5d0b2ff > > ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/configuration/zeppelin-env.xml > e6c59d5 > > ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5/configuration/zookeeper-env.xml > 696b28a > ambari-server/src/main/resources/configuration-schema.xsd daba29f > > Diff: https://reviews.apache.org/r/53213/diff/ > > > Testing > --- > > Tested manually. Suitable keytabs are generated automatically when not > mentioned in blueprint. Service starts succeeded with a blueprint exported > with the code change. > Modified existing unit test cases. > > > Thanks, > > Amruta Borkar > >
Re: Review Request 53213: Exporting blueprint from kerberos enabled cluster, exports hardcoded values cluster name and realm in principal_name property
> On Jan. 18, 2017, 6:02 p.m., Di Li wrote: > > Ship It! Thank you Di, could you please help me push this to trunk? - Amruta --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/53213/#review162146 --- On Jan. 12, 2017, 12:33 a.m., Amruta Borkar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/53213/ > --- > > (Updated Jan. 12, 2017, 12:33 a.m.) > > > Review request for Ambari, Di Li, Jaimin Jetly, Jayush Luniya, Robert Levas, > and Robert Nettleton. > > > Bugs: AMBARI-18692 > https://issues.apache.org/jira/browse/AMBARI-18692 > > > Repository: ambari > > > Description > --- > > Exporting blueprint from kerberos enabled cluster, exports hardcoded values > cluster name and realm in principal_name property. > When the same blueprint is used to create another cluster with different > name, service start fail with following error: > "resource_management.core.exceptions.Fail: Execution of '/usr/bin/kinit -kt > /etc/security/keytabs/hdfs.headless.keytab [keytab_name_in_blueprint] eturned > 1. kinit: Keytab contains no suitable keys for [keytab_name_in_blueprint] > while getting initial credentials" > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java > 737e8de > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/Stack.java > 16f75ee > > ambari-server/src/main/java/org/apache/ambari/server/state/PropertyInfo.java > 01c8a89 > > ambari-server/src/main/resources/common-services/ACCUMULO/1.6.1.2.2.0/configuration/accumulo-env.xml > 1d330dd > > ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/configuration/infra-solr-env.xml > d7ae236 > > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-hbase-security-site.xml > 2be101b > > ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/configuration/application-properties.xml > 9efa4f9 > > ambari-server/src/main/resources/common-services/DRUID/0.9.2/configuration/druid-common.xml > 0ded5d6 > > ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/configuration/hbase-env.xml > 2ab6aa0 > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/configuration/hadoop-env.xml > c2f37c1 > > ambari-server/src/main/resources/common-services/KAFKA/0.8.1/configuration/kafka-env.xml > e8a76b6 > > ambari-server/src/main/resources/common-services/KERBEROS/1.10.3-10/configuration/kerberos-env.xml > 7016437 > > ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/configuration/knox-env.xml > 2e5a026 > > ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logfeeder-env.xml > fb641a8 > > ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-env.xml > a6f1502 > > ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-properties.xml > 47872ee > > ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-admin-site.xml > 36ebc8c > > ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-tagsync-site.xml > a0bd322 > > ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-ugsync-site.xml > d267b75 > > ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/configuration/kms-site.xml > 1bce4e6 > > ambari-server/src/main/resources/common-services/SPARK/1.2.1/configuration/spark-defaults.xml > b85cf50 > > ambari-server/src/main/resources/common-services/SPARK/1.2.1/configuration/spark-env.xml > 71a42f7 > > ambari-server/src/main/resources/common-services/SPARK2/2.0.0/configuration/spark2-defaults.xml > a5115af > > ambari-server/src/main/resources/common-services/SPARK2/2.0.0/configuration/spark2-env.xml > b78fd79 > > ambari-server/src/main/resources/common-services/STORM/0.9.1/configuration/storm-env.xml > 5d0b2ff > > ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/configuration/zeppelin-env.xml > e6c59d5 > > ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5/configuration/zookeeper-env.xml > 696b28a > ambari-server/sr
Re: Review Request 53213: Exporting blueprint from kerberos enabled cluster, exports hardcoded values cluster name and realm in principal_name property
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/53213/ --- (Updated Jan. 12, 2017, 12:33 a.m.) Review request for Ambari, Di Li, Jaimin Jetly, Jayush Luniya, Robert Levas, and Robert Nettleton. Changes --- Updated the patch to fit the latest trunk code. Bugs: AMBARI-18692 https://issues.apache.org/jira/browse/AMBARI-18692 Repository: ambari Description --- Exporting blueprint from kerberos enabled cluster, exports hardcoded values cluster name and realm in principal_name property. When the same blueprint is used to create another cluster with different name, service start fail with following error: "resource_management.core.exceptions.Fail: Execution of '/usr/bin/kinit -kt /etc/security/keytabs/hdfs.headless.keytab [keytab_name_in_blueprint] eturned 1. kinit: Keytab contains no suitable keys for [keytab_name_in_blueprint] while getting initial credentials" Diffs (updated) - ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java 737e8de ambari-server/src/main/java/org/apache/ambari/server/controller/internal/Stack.java 16f75ee ambari-server/src/main/java/org/apache/ambari/server/state/PropertyInfo.java 01c8a89 ambari-server/src/main/resources/common-services/ACCUMULO/1.6.1.2.2.0/configuration/accumulo-env.xml 1d330dd ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/configuration/infra-solr-env.xml d7ae236 ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-hbase-security-site.xml 2be101b ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/configuration/application-properties.xml 9efa4f9 ambari-server/src/main/resources/common-services/DRUID/0.9.2/configuration/druid-common.xml 0ded5d6 ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/configuration/hbase-env.xml 2ab6aa0 ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/configuration/hadoop-env.xml c2f37c1 ambari-server/src/main/resources/common-services/KAFKA/0.8.1/configuration/kafka-env.xml e8a76b6 ambari-server/src/main/resources/common-services/KERBEROS/1.10.3-10/configuration/kerberos-env.xml 7016437 ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/configuration/knox-env.xml 2e5a026 ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logfeeder-env.xml fb641a8 ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-env.xml a6f1502 ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-properties.xml 47872ee ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-admin-site.xml 36ebc8c ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-tagsync-site.xml a0bd322 ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-ugsync-site.xml d267b75 ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/configuration/kms-site.xml 1bce4e6 ambari-server/src/main/resources/common-services/SPARK/1.2.1/configuration/spark-defaults.xml b85cf50 ambari-server/src/main/resources/common-services/SPARK/1.2.1/configuration/spark-env.xml 71a42f7 ambari-server/src/main/resources/common-services/SPARK2/2.0.0/configuration/spark2-defaults.xml a5115af ambari-server/src/main/resources/common-services/SPARK2/2.0.0/configuration/spark2-env.xml b78fd79 ambari-server/src/main/resources/common-services/STORM/0.9.1/configuration/storm-env.xml 5d0b2ff ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/configuration/zeppelin-env.xml e6c59d5 ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5/configuration/zookeeper-env.xml 696b28a ambari-server/src/main/resources/configuration-schema.xsd daba29f Diff: https://reviews.apache.org/r/53213/diff/ Testing --- Tested manually. Suitable keytabs are generated automatically when not mentioned in blueprint. Service starts succeeded with a blueprint exported with the code change. Modified existing unit test cases. Thanks, Amruta Borkar
Re: Review Request 54259: In stack version, build number should not be mandatory
> On Dec. 2, 2016, 7:16 p.m., Di Li wrote: > > Ship It! Thanks Di, Could you please help me push this to trunk? - Amruta --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/54259/#review157793 --- On Dec. 2, 2016, 5:44 p.m., Amruta Borkar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/54259/ > --- > > (Updated Dec. 2, 2016, 5:44 p.m.) > > > Review request for Ambari, Alejandro Fernandez and Di Li. > > > Bugs: AMBARI-19039 > https://issues.apache.org/jira/browse/AMBARI-19039 > > > Repository: ambari > > > Description > --- > > In stack version, build number should not be mandatory. mapreduce.tar.gz file > does not get copied during blueprint deployment when stack version does not > have build version. > > > Diffs > - > > > ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py > 519c88b > > ambari-common/src/main/python/resource_management/libraries/functions/dynamic_variable_interpretation.py > ca8fe19 > > Diff: https://reviews.apache.org/r/54259/diff/ > > > Testing > --- > > Test manually with stack version with build number and without build number. > > > Thanks, > > Amruta Borkar > >
Review Request 54259: In stack version, build number should not be mandatory
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/54259/ --- Review request for Ambari, Alejandro Fernandez and Di Li. Bugs: AMBARI-19039 https://issues.apache.org/jira/browse/AMBARI-19039 Repository: ambari Description --- In stack version, build number should not be mandatory. mapreduce.tar.gz file does not get copied during blueprint deployment when stack version does not have build version. Diffs - ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py 519c88b ambari-common/src/main/python/resource_management/libraries/functions/dynamic_variable_interpretation.py ca8fe19 Diff: https://reviews.apache.org/r/54259/diff/ Testing --- Test manually with stack version with build number and without build number. Thanks, Amruta Borkar
Re: Review Request 51815: Introduce conditional dependencies in stack defition to handle blueprint validation gracefully
> On Nov. 10, 2016, 4:22 p.m., Nate Cole wrote: > > Ship It! Thank you Nate, could you please help me push the patch to trunk? I do not have that access. - Amruta --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51815/#review155603 --- On Nov. 1, 2016, 6:57 p.m., Amruta Borkar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/51815/ > --- > > (Updated Nov. 1, 2016, 6:57 p.m.) > > > Review request for Ambari, Shantanu Mundkur, Di Li, Jayush Luniya, Juanjo > Marron, Laszlo Puskas, Nate Cole, and Robert Nettleton. > > > Bugs: AMBARI-18355 > https://issues.apache.org/jira/browse/AMBARI-18355 > > > Repository: ambari > > > Description > --- > > Currently stack definitions do not list conditional dependencies, adding > those to the stack definitions would make it easy to validate errors in case > of blueprint deployment. Please refer to document attached to Jira > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/state/DependencyConditionInfo.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/state/DependencyInfo.java > e3db662 > > ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintValidatorImpl.java > a5f33ff > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml > 9c1387d > > ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintImplTest.java > ff9af17 > > ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintValidatorImplTest.java > b1de8ef > > Diff: https://reviews.apache.org/r/51815/diff/ > > > Testing > --- > > Written Junit test cases. Perfomred manual testing to check the Namenode HA > component dependency. Was able to proceed with the installation for valid > blueprint. and got validation error message while registering blueprint when > the Blueprint did not satisfy the conditional dependencies. > > > Thanks, > > Amruta Borkar > >
Re: Review Request 51815: Introduce conditional dependencies in stack defition to handle blueprint validation gracefully
> On Oct. 13, 2016, 1:24 p.m., Nate Cole wrote: > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml, > > lines 43-47 > > <https://reviews.apache.org/r/51815/diff/5/?file=1533651#file1533651line43> > > > > This is a bit of an odd syntax. the element should be a > > different class then you don't need the element. Now, > > these metainfo.xml files don't have schema for validation, but java XML is > > able to use correct classes based on the element name. So for example, you > > would have two types such that: > > > > > > > > > > > > > > > > > > > > > > Hello Nate, Could you please provide your feedback on the updated patch? Thank you, Amruta - Amruta --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51815/#review152489 --- On Nov. 1, 2016, 6:57 p.m., Amruta Borkar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/51815/ > --- > > (Updated Nov. 1, 2016, 6:57 p.m.) > > > Review request for Ambari, Shantanu Mundkur, Di Li, Jayush Luniya, Juanjo > Marron, Laszlo Puskas, Nate Cole, and Robert Nettleton. > > > Bugs: AMBARI-18355 > https://issues.apache.org/jira/browse/AMBARI-18355 > > > Repository: ambari > > > Description > --- > > Currently stack definitions do not list conditional dependencies, adding > those to the stack definitions would make it easy to validate errors in case > of blueprint deployment. Please refer to document attached to Jira > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/state/DependencyConditionInfo.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/state/DependencyInfo.java > e3db662 > > ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintValidatorImpl.java > a5f33ff > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml > 9c1387d > > ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintImplTest.java > ff9af17 > > ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintValidatorImplTest.java > b1de8ef > > Diff: https://reviews.apache.org/r/51815/diff/ > > > Testing > --- > > Written Junit test cases. Perfomred manual testing to check the Namenode HA > component dependency. Was able to proceed with the installation for valid > blueprint. and got validation error message while registering blueprint when > the Blueprint did not satisfy the conditional dependencies. > > > Thanks, > > Amruta Borkar > >
Re: Review Request 53213: Exporting blueprint from kerberos enabled cluster, exports hardcoded values cluster name and realm in principal_name property
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/53213/ --- (Updated Nov. 9, 2016, 12:11 a.m.) Review request for Ambari, Di Li, Jaimin Jetly, Jayush Luniya, Robert Levas, and Robert Nettleton. Bugs: AMBARI-18692 https://issues.apache.org/jira/browse/AMBARI-18692 Repository: ambari Description --- Exporting blueprint from kerberos enabled cluster, exports hardcoded values cluster name and realm in principal_name property. When the same blueprint is used to create another cluster with different name, service start fail with following error: "resource_management.core.exceptions.Fail: Execution of '/usr/bin/kinit -kt /etc/security/keytabs/hdfs.headless.keytab [keytab_name_in_blueprint] eturned 1. kinit: Keytab contains no suitable keys for [keytab_name_in_blueprint] while getting initial credentials" Diffs (updated) - ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java f890326 ambari-server/src/main/java/org/apache/ambari/server/controller/internal/Stack.java 16f75ee ambari-server/src/main/java/org/apache/ambari/server/state/PropertyInfo.java 81de76c ambari-server/src/main/resources/common-services/ACCUMULO/1.6.1.2.2.0/configuration/accumulo-env.xml 1d330dd ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/configuration/infra-solr-env.xml d7ae236 ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-hbase-security-site.xml 2be101b ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/configuration/application-properties.xml 9efa4f9 ambari-server/src/main/resources/common-services/DRUID/0.9.2/configuration/druid-common.xml 8712c7d ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/configuration/hbase-env.xml 24bd563 ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/configuration/hadoop-env.xml afaaee8 ambari-server/src/main/resources/common-services/KAFKA/0.8.1/configuration/kafka-env.xml e8a76b6 ambari-server/src/main/resources/common-services/KERBEROS/1.10.3-10/configuration/kerberos-env.xml 7016437 ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/configuration/knox-env.xml 2e5a026 ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logfeeder-env.xml ee885e3 ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-env.xml c5b9b4e ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-admin-site.xml 36ebc8c ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-tagsync-site.xml a0bd322 ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-ugsync-site.xml d267b75 ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/configuration/kms-site.xml 1bce4e6 ambari-server/src/main/resources/common-services/SPARK/1.2.1/configuration/spark-defaults.xml b85cf50 ambari-server/src/main/resources/common-services/SPARK/1.2.1/configuration/spark-env.xml 71a42f7 ambari-server/src/main/resources/common-services/SPARK2/2.0.0/configuration/spark2-defaults.xml a5115af ambari-server/src/main/resources/common-services/SPARK2/2.0.0/configuration/spark2-env.xml b78fd79 ambari-server/src/main/resources/common-services/STORM/0.9.1/configuration/storm-env.xml 5d0b2ff ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/configuration/zeppelin-env.xml c03d2dc ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5/configuration/zookeeper-env.xml 696b28a ambari-server/src/main/resources/configuration-schema.xsd daba29f ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java 5bedb9d Diff: https://reviews.apache.org/r/53213/diff/ Testing --- Tested manually. Suitable keytabs are generated automatically when not mentioned in blueprint. Service starts succeeded with a blueprint exported with the code change. Modified existing unit test cases. Thanks, Amruta Borkar
Re: Review Request 53213: Exporting blueprint from kerberos enabled cluster, exports hardcoded values cluster name and realm in principal_name property
> On Oct. 31, 2016, 3:31 p.m., Robert Nettleton wrote: > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java, > > line 2993 > > <https://reviews.apache.org/r/53213/diff/1/?file=1546962#file1546962line2993> > > > > Are we sure that all properties that include the string "principal" > > should be excluded? > > > > Are all of these properties certain to have the cluster-specific name > > that would be an issue for portability? > > > > Given the general nature of this property name matcher, I'd be a little > > concerned that maybe more is being filtered out than necessary. > > > > I'm not sure that this is definitely a problem, but it might be a good > > idea to double-check the stack definitions, to make sure that only the > > problematic properties are being excluded. Changed the patch according to Robert's suggestions, which would avoid this issue. - Amruta --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/53213/#review154268 --- On Oct. 28, 2016, 5:50 p.m., Amruta Borkar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/53213/ > --- > > (Updated Oct. 28, 2016, 5:50 p.m.) > > > Review request for Ambari, Di Li, Robert Levas, and Robert Nettleton. > > > Bugs: AMBARI-18692 > https://issues.apache.org/jira/browse/AMBARI-18692 > > > Repository: ambari > > > Description > --- > > Exporting blueprint from kerberos enabled cluster, exports hardcoded values > cluster name and realm in principal_name property. > When the same blueprint is used to create another cluster with different > name, service start fail with following error: > "resource_management.core.exceptions.Fail: Execution of '/usr/bin/kinit -kt > /etc/security/keytabs/hdfs.headless.keytab [keytab_name_in_blueprint] eturned > 1. kinit: Keytab contains no suitable keys for [keytab_name_in_blueprint] > while getting initial credentials" > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java > f890326 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java > 5bedb9d > > Diff: https://reviews.apache.org/r/53213/diff/ > > > Testing > --- > > Tested manually. Suitable keytabs are generated automatically when not > mentioned in blueprint. Service starts succeeded with a blueprint exported > with the code change. > Modified existing unit test cases. > > > Thanks, > > Amruta Borkar > >
Re: Review Request 53213: Exporting blueprint from kerberos enabled cluster, exports hardcoded values cluster name and realm in principal_name property
> On Oct. 31, 2016, 4:27 p.m., Robert Levas wrote: > > This seems dangerous. > > > > I think it would be a better option to make this explicit by using a > > configuration attribute. For example: > > > > ''' > > > > ... > > kerberos_principal > > ... > > > > ''' > > > > However I am not sure if this data is available at the time you would need > > it. > > Amruta Borkar wrote: > Hello Robert, > Would it be ok if a new property type is defined to identify kerberos > principal EX: kerberos-principal rather than > defining it in ? As we currently use > password to identify and filter out password > references while blueprint export. > > Robert Levas wrote: > I think that will work too but we might need some more expertice on this. > After making the change, can you add Jayush Luniya and Jaimin Jetly to the > review? Also, make sure you update `configuration-schema.xsd` and > `org.apache.ambari.server.state.PropertyInfo.PropertyType` Updated the the patch based on suggestions. Also adding Jayush Luniya and Jaimin Jetly. - Amruta --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/53213/#review154274 --- On Oct. 28, 2016, 5:50 p.m., Amruta Borkar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/53213/ > --- > > (Updated Oct. 28, 2016, 5:50 p.m.) > > > Review request for Ambari, Di Li, Robert Levas, and Robert Nettleton. > > > Bugs: AMBARI-18692 > https://issues.apache.org/jira/browse/AMBARI-18692 > > > Repository: ambari > > > Description > --- > > Exporting blueprint from kerberos enabled cluster, exports hardcoded values > cluster name and realm in principal_name property. > When the same blueprint is used to create another cluster with different > name, service start fail with following error: > "resource_management.core.exceptions.Fail: Execution of '/usr/bin/kinit -kt > /etc/security/keytabs/hdfs.headless.keytab [keytab_name_in_blueprint] eturned > 1. kinit: Keytab contains no suitable keys for [keytab_name_in_blueprint] > while getting initial credentials" > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java > f890326 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java > 5bedb9d > > Diff: https://reviews.apache.org/r/53213/diff/ > > > Testing > --- > > Tested manually. Suitable keytabs are generated automatically when not > mentioned in blueprint. Service starts succeeded with a blueprint exported > with the code change. > Modified existing unit test cases. > > > Thanks, > > Amruta Borkar > >
Re: Review Request 53213: Exporting blueprint from kerberos enabled cluster, exports hardcoded values cluster name and realm in principal_name property
> On Oct. 31, 2016, 4:27 p.m., Robert Levas wrote: > > This seems dangerous. > > > > I think it would be a better option to make this explicit by using a > > configuration attribute. For example: > > > > ''' > > > > ... > > kerberos_principal > > ... > > > > ''' > > > > However I am not sure if this data is available at the time you would need > > it. Hello Robert, Would it be ok if a new property type is defined to identify kerberos principal EX: kerberos-principal rather than defining it in ? As we currently use password to identify and filter out password references while blueprint export. - Amruta --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/53213/#review154274 --- On Oct. 28, 2016, 5:50 p.m., Amruta Borkar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/53213/ > --- > > (Updated Oct. 28, 2016, 5:50 p.m.) > > > Review request for Ambari, Di Li, Robert Levas, and Robert Nettleton. > > > Bugs: AMBARI-18692 > https://issues.apache.org/jira/browse/AMBARI-18692 > > > Repository: ambari > > > Description > --- > > Exporting blueprint from kerberos enabled cluster, exports hardcoded values > cluster name and realm in principal_name property. > When the same blueprint is used to create another cluster with different > name, service start fail with following error: > "resource_management.core.exceptions.Fail: Execution of '/usr/bin/kinit -kt > /etc/security/keytabs/hdfs.headless.keytab [keytab_name_in_blueprint] eturned > 1. kinit: Keytab contains no suitable keys for [keytab_name_in_blueprint] > while getting initial credentials" > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java > f890326 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java > 5bedb9d > > Diff: https://reviews.apache.org/r/53213/diff/ > > > Testing > --- > > Tested manually. Suitable keytabs are generated automatically when not > mentioned in blueprint. Service starts succeeded with a blueprint exported > with the code change. > Modified existing unit test cases. > > > Thanks, > > Amruta Borkar > >
Re: Review Request 51815: Introduce conditional dependencies in stack defition to handle blueprint validation gracefully
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51815/ --- (Updated Nov. 1, 2016, 6:57 p.m.) Review request for Ambari, Shantanu Mundkur, Di Li, Jayush Luniya, Juanjo Marron, Laszlo Puskas, Nate Cole, and Robert Nettleton. Changes --- Hello Nate, Could you please review the updated patch. Thank you Bugs: AMBARI-18355 https://issues.apache.org/jira/browse/AMBARI-18355 Repository: ambari Description --- Currently stack definitions do not list conditional dependencies, adding those to the stack definitions would make it easy to validate errors in case of blueprint deployment. Please refer to document attached to Jira Diffs (updated) - ambari-server/src/main/java/org/apache/ambari/server/state/DependencyConditionInfo.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/state/DependencyInfo.java e3db662 ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintValidatorImpl.java a5f33ff ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml 9c1387d ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintImplTest.java ff9af17 ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintValidatorImplTest.java b1de8ef Diff: https://reviews.apache.org/r/51815/diff/ Testing --- Written Junit test cases. Perfomred manual testing to check the Namenode HA component dependency. Was able to proceed with the installation for valid blueprint. and got validation error message while registering blueprint when the Blueprint did not satisfy the conditional dependencies. Thanks, Amruta Borkar
Review Request 53213: Exporting blueprint from kerberos enabled cluster, exports hardcoded values cluster name and realm in principal_name property
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/53213/ --- Review request for Ambari, Robert Levas and Robert Nettleton. Bugs: AMBARI-18692 https://issues.apache.org/jira/browse/AMBARI-18692 Repository: ambari Description --- Exporting blueprint from kerberos enabled cluster, exports hardcoded values cluster name and realm in principal_name property. When the same blueprint is used to create another cluster with different name, service start fail with following error: "resource_management.core.exceptions.Fail: Execution of '/usr/bin/kinit -kt /etc/security/keytabs/hdfs.headless.keytab [keytab_name_in_blueprint] eturned 1. kinit: Keytab contains no suitable keys for [keytab_name_in_blueprint] while getting initial credentials" Diffs - ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java f890326 ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java 5bedb9d Diff: https://reviews.apache.org/r/53213/diff/ Testing --- Tested manually. Suitable keytabs are generated automatically when not mentioned in blueprint. Service starts succeeded with a blueprint exported with the code change. Modified existing unit test cases. Thanks, Amruta Borkar
Re: Review Request 51815: Introduce conditional dependencies in stack defition to handle blueprint validation gracefully
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51815/ --- (Updated Oct. 12, 2016, 11:42 p.m.) Review request for Ambari, Shantanu Mundkur, Di Li, Jayush Luniya, Juanjo Marron, Laszlo Puskas, Nate Cole, and Robert Nettleton. Changes --- Hello Nate, I have modified the patch according to your comments. Could you please take a look? Thank you Bugs: AMBARI-18355 https://issues.apache.org/jira/browse/AMBARI-18355 Repository: ambari Description --- Currently stack definitions do not list conditional dependencies, adding those to the stack definitions would make it easy to validate errors in case of blueprint deployment. Please refer to document attached to Jira Diffs (updated) - ambari-server/src/main/java/org/apache/ambari/server/state/DependencyConditionAdapter.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/state/DependencyConditionInfo.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/state/DependencyInfo.java e3db662 ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintValidatorImpl.java a5f33ff ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml 65d166a ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintImplTest.java ff9af17 ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintValidatorImplTest.java b1de8ef Diff: https://reviews.apache.org/r/51815/diff/ Testing --- Written Junit test cases. Perfomred manual testing to check the Namenode HA component dependency. Was able to proceed with the installation for valid blueprint. and got validation error message while registering blueprint when the Blueprint did not satisfy the conditional dependencies. Thanks, Amruta Borkar
Re: Review Request 51815: Introduce conditional dependencies in stack defition to handle blueprint validation gracefully
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51815/ --- (Updated Sept. 30, 2016, 12:39 a.m.) Review request for Ambari, Shantanu Mundkur, Di Li, Jayush Luniya, Juanjo Marron, Laszlo Puskas, and Robert Nettleton. Bugs: AMBARI-18355 https://issues.apache.org/jira/browse/AMBARI-18355 Repository: ambari Description --- Currently stack definitions do not list conditional dependencies, adding those to the stack definitions would make it easy to validate errors in case of blueprint deployment. Please refer to document attached to Jira Diffs (updated) - ambari-server/src/main/java/org/apache/ambari/server/state/DependencyConditionInfo.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/state/DependencyInfo.java e3db662 ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintValidatorImpl.java a5f33ff ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml 65d166a ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintValidatorImplTest.java b1de8ef Diff: https://reviews.apache.org/r/51815/diff/ Testing --- Written Junit test cases. Perfomred manual testing to check the Namenode HA component dependency. Was able to proceed with the installation for valid blueprint. and got validation error message while registering blueprint when the Blueprint did not satisfy the conditional dependencies. Thanks, Amruta Borkar
Re: Review Request 51815: Introduce conditional dependencies in stack defition to handle blueprint validation gracefully
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51815/ --- (Updated Sept. 20, 2016, 11:18 p.m.) Review request for Ambari, Shantanu Mundkur, Di Li, Jayush Luniya, Juanjo Marron, Laszlo Puskas, and Robert Nettleton. Changes --- Hello, I have added tag to represent different condition-types and made two sub-types 'IF-PROPERTY-EXISTS' and 'PROPERTY-VALUE-EQUALS'. Also addressed other two changes suggested by Robert. I am still working on adding more test cases. Please review and provide feedback. Bugs: AMBARI-18355 https://issues.apache.org/jira/browse/AMBARI-18355 Repository: ambari Description --- Currently stack definitions do not list conditional dependencies, adding those to the stack definitions would make it easy to validate errors in case of blueprint deployment. Please refer to document attached to Jira Diffs (updated) - ambari-server/src/main/java/org/apache/ambari/server/state/DependencyConditionInfo.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/state/DependencyInfo.java e3db662 ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintValidatorImpl.java a5f33ff ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml 65d166a ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintValidatorImplTest.java b1de8ef Diff: https://reviews.apache.org/r/51815/diff/ Testing --- Written Junit test cases. Perfomred manual testing to check the Namenode HA component dependency. Was able to proceed with the installation for valid blueprint. and got validation error message while registering blueprint when the Blueprint did not satisfy the conditional dependencies. Thanks, Amruta Borkar
Re: Review Request 51815: Introduce conditional dependencies in stack defition to handle blueprint validation gracefully
> On Sept. 14, 2016, 5:11 p.m., Di Li wrote: > > ambari-server/src/main/java/org/apache/ambari/server/state/DependencyConditionInfo.java, > > line 35 > > <https://reviews.apache.org/r/51815/diff/2/?file=1497961#file1497961line35> > > > > should configType, property, value be mapped to xml elements explicitly > > ? Hi Di, In this case mapping should not be necessary as properties in class have same names as defined in metainfo.xml and DependencyConditions class is mapped to xml element 'condition' in DependencyInfo class. - Amruta --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51815/#review148930 ------- On Sept. 13, 2016, 9:28 p.m., Amruta Borkar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/51815/ > --- > > (Updated Sept. 13, 2016, 9:28 p.m.) > > > Review request for Ambari, Shantanu Mundkur, Di Li, Juanjo Marron, Laszlo > Puskas, and Robert Nettleton. > > > Bugs: AMBARI-18355 > https://issues.apache.org/jira/browse/AMBARI-18355 > > > Repository: ambari > > > Description > --- > > Currently stack definitions do not list conditional dependencies, adding > those to the stack definitions would make it easy to validate errors in case > of blueprint deployment. Please refer to document attached to Jira > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/state/DependencyConditionInfo.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/state/DependencyInfo.java > e3db662 > > ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintValidatorImpl.java > a5f33ff > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml > 65d166a > > ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintValidatorImplTest.java > b1de8ef > > Diff: https://reviews.apache.org/r/51815/diff/ > > > Testing > --- > > Written Junit test cases. Perfomred manual testing to check the Namenode HA > component dependency. Was able to proceed with the installation for valid > blueprint. and got validation error message while registering blueprint when > the Blueprint did not satisfy the conditional dependencies. > > > Thanks, > > Amruta Borkar > >
Re: Review Request 51815: Introduce conditional dependencies in stack defition to handle blueprint validation gracefully
> On Sept. 13, 2016, 2:07 p.m., Di Li wrote: > > ambari-server/src/main/java/org/apache/ambari/server/state/DependencyConditionInfo.java, > > line 103 > > <https://reviews.apache.org/r/51815/diff/1/?file=1496958#file1496958line103> > > > > do you support other type of checks ? such as the property has to be a > > certain value ? Hi Di, Yes, property value check is handled by the tag as given in the document. This tag is optional and if it is present, then the blueprint will be validated for the the mentioned in metainfo.xml file. This check is performed by following code in isResolved() method: //if 'propertyValue' is null then it is assumed that condition only checks if the 'property' exists or not if(propertyValue == null || propertyValue.equals(properties.get(configType).get(property))) return true; For this scenario (NAMENODE HA) we do not have conditional dependency based on propertyValue hence I have not added it in the metainfo.xml file. > On Sept. 13, 2016, 2:07 p.m., Di Li wrote: > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml, > > line 58 > > <https://reviews.apache.org/r/51815/diff/1/?file=1496961#file1496961line58> > > > > what does "dfs.nameservices" mean in this > > conditional check? as a negative test, what if a user manually added the > > property to hdfs-site.xml ? When a strucutre like .. ... is present in metainfo.xml file code will look to see if this property is present in blueprint. If not then the will not be considered mandatory. I have modified the patch to consider configurations provided in hdfs-site.xml. > On Sept. 13, 2016, 2:07 p.m., Di Li wrote: > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml, > > line 55 > > <https://reviews.apache.org/r/51815/diff/1/?file=1496961#file1496961line55> > > > > this is already a subsection of "dependency", it can just be called > > "conditions" Modified the patch. > On Sept. 13, 2016, 2:07 p.m., Di Li wrote: > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml, > > line 56 > > <https://reviews.apache.org/r/51815/diff/1/?file=1496961#file1496961line56> > > > > same here, no need to reiterate it's for "dependency" Modified the patch. - Amruta ------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51815/#review148721 --- On Sept. 13, 2016, 9:28 p.m., Amruta Borkar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/51815/ > --- > > (Updated Sept. 13, 2016, 9:28 p.m.) > > > Review request for Ambari, Shantanu Mundkur, Di Li, Juanjo Marron, Laszlo > Puskas, and Robert Nettleton. > > > Bugs: AMBARI-18355 > https://issues.apache.org/jira/browse/AMBARI-18355 > > > Repository: ambari > > > Description > --- > > Currently stack definitions do not list conditional dependencies, adding > those to the stack definitions would make it easy to validate errors in case > of blueprint deployment. Please refer to document attached to Jira > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/state/DependencyConditionInfo.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/state/DependencyInfo.java > e3db662 > > ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintValidatorImpl.java > a5f33ff > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml > 65d166a > > ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintValidatorImplTest.java > b1de8ef > > Diff: https://reviews.apache.org/r/51815/diff/ > > > Testing > --- > > Written Junit test cases. Perfomred manual testing to check the Namenode HA > component dependency. Was able to proceed with the installation for valid > blueprint. and got validation error message while registering blueprint when > the Blueprint did not satisfy the conditional dependencies. > > > Thanks, > > Amruta Borkar > >
Re: Review Request 51815: Introduce conditional dependencies in stack defition to handle blueprint validation gracefully
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51815/ --- (Updated Sept. 13, 2016, 9:28 p.m.) Review request for Ambari, Shantanu Mundkur, Di Li, Juanjo Marron, Laszlo Puskas, and Robert Nettleton. Bugs: AMBARI-18355 https://issues.apache.org/jira/browse/AMBARI-18355 Repository: ambari Description --- Currently stack definitions do not list conditional dependencies, adding those to the stack definitions would make it easy to validate errors in case of blueprint deployment. Please refer to document attached to Jira Diffs (updated) - ambari-server/src/main/java/org/apache/ambari/server/state/DependencyConditionInfo.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/state/DependencyInfo.java e3db662 ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintValidatorImpl.java a5f33ff ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml 65d166a ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintValidatorImplTest.java b1de8ef Diff: https://reviews.apache.org/r/51815/diff/ Testing --- Written Junit test cases. Perfomred manual testing to check the Namenode HA component dependency. Was able to proceed with the installation for valid blueprint. and got validation error message while registering blueprint when the Blueprint did not satisfy the conditional dependencies. Thanks, Amruta Borkar
Review Request 51815: Introduce conditional dependencies in stack defition to handle blueprint validation gracefully
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51815/ --- Review request for Ambari, Di Li, Juanjo Marron, Laszlo Puskas, and Robert Nettleton. Bugs: AMBARI-18355 https://issues.apache.org/jira/browse/AMBARI-18355 Repository: ambari Description --- Currently stack definitions do not list conditional dependencies, adding those to the stack definitions would make it easy to validate errors in case of blueprint deployment. Please refer to document attached to Jira Diffs - ambari-server/src/main/java/org/apache/ambari/server/state/DependencyConditionInfo.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/state/DependencyInfo.java e3db662 ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintValidatorImpl.java a5f33ff ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml 65d166a ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintValidatorImplTest.java b1de8ef Diff: https://reviews.apache.org/r/51815/diff/ Testing --- Written Junit test cases. Perfomred manual testing to check the Namenode HA component dependency. Was able to proceed with the installation for valid blueprint. and got validation error message while registering blueprint when the Blueprint did not satisfy the conditional dependencies. Thanks, Amruta Borkar
Re: Review Request 51708: While checking for component dependency code looks for incorrect component name
> On Sept. 8, 2016, 4:06 p.m., Robert Nettleton wrote: > > The patch looks fine to me. > > > > Thanks for catching and fixing this bug. > > Amruta Borkar wrote: > Thank you Robert, could you please help me push this to trunk? > > Robert Nettleton wrote: > Hi Amruta, > > Sure, I can push this to trunk today. > > Robert Nettleton wrote: > Hi Amruta, > > I've pushed this patch into trunk. > > Can you please close out the review and the associated JIRA? > > Thanks. Ok. Thank you. - Amruta --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51708/#review148198 ------- On Sept. 7, 2016, 10:03 p.m., Amruta Borkar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/51708/ > --- > > (Updated Sept. 7, 2016, 10:03 p.m.) > > > Review request for Ambari, Laszlo Puskas and Robert Nettleton. > > > Bugs: AMBARI-18333 > https://issues.apache.org/jira/browse/AMBARI-18333 > > > Repository: ambari > > > Description > --- > > While validating host groups in blueprint for component dependency, code > should check if a dependency component is present in the host group component > list rather than the component on whose dependency list it is iterating. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintValidatorImpl.java > 398772e > > ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintValidatorImplTest.java > ea6bee3 > > Diff: https://reviews.apache.org/r/51708/diff/ > > > Testing > --- > > Added Test cases, performed manual testing > > > Thanks, > > Amruta Borkar > >
Re: Review Request 51708: While checking for component dependency code looks for incorrect component name
> On Sept. 8, 2016, 4:06 p.m., Robert Nettleton wrote: > > The patch looks fine to me. > > > > Thanks for catching and fixing this bug. Thank you Robert, could you please help me push this to trunk? - Amruta --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51708/#review148198 --- On Sept. 7, 2016, 10:03 p.m., Amruta Borkar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/51708/ > --- > > (Updated Sept. 7, 2016, 10:03 p.m.) > > > Review request for Ambari, Laszlo Puskas and Robert Nettleton. > > > Bugs: AMBARI-18333 > https://issues.apache.org/jira/browse/AMBARI-18333 > > > Repository: ambari > > > Description > --- > > While validating host groups in blueprint for component dependency, code > should check if a dependency component is present in the host group component > list rather than the component on whose dependency list it is iterating. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintValidatorImpl.java > 398772e > > ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintValidatorImplTest.java > ea6bee3 > > Diff: https://reviews.apache.org/r/51708/diff/ > > > Testing > --- > > Added Test cases, performed manual testing > > > Thanks, > > Amruta Borkar > >
Re: Review Request 51708: While checking for component dependency code looks for incorrect component name
> On Sept. 8, 2016, 8:03 a.m., Laszlo Puskas wrote: > > Ship It! Thank you Laszlo. - Amruta --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51708/#review148157 --- On Sept. 7, 2016, 10:03 p.m., Amruta Borkar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/51708/ > --- > > (Updated Sept. 7, 2016, 10:03 p.m.) > > > Review request for Ambari, Laszlo Puskas and Robert Nettleton. > > > Bugs: AMBARI-18333 > https://issues.apache.org/jira/browse/AMBARI-18333 > > > Repository: ambari > > > Description > --- > > While validating host groups in blueprint for component dependency, code > should check if a dependency component is present in the host group component > list rather than the component on whose dependency list it is iterating. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintValidatorImpl.java > 398772e > > ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintValidatorImplTest.java > ea6bee3 > > Diff: https://reviews.apache.org/r/51708/diff/ > > > Testing > --- > > Added Test cases, performed manual testing > > > Thanks, > > Amruta Borkar > >
Review Request 51708: While checking for component dependency code looks for incorrect component name
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51708/ --- Review request for Ambari, Laszlo Puskas and Robert Nettleton. Bugs: AMBARI-18333 https://issues.apache.org/jira/browse/AMBARI-18333 Repository: ambari Description --- While validating host groups in blueprint for component dependency, code should check if a dependency component is present in the host group component list rather than the component on whose dependency list it is iterating. Diffs - ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintValidatorImpl.java 398772e ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintValidatorImplTest.java ea6bee3 Diff: https://reviews.apache.org/r/51708/diff/ Testing --- Added Test cases, performed manual testing Thanks, Amruta Borkar
Re: Review Request 51055: Avoid hardcoded values in blueprint for hive and oozie
> On Aug. 15, 2016, 5:28 p.m., Di Li wrote: > > Ship It! Thank you Di. Could you please help me push this to trunk? - Amruta --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51055/#review145773 --- On Aug. 15, 2016, 11:54 p.m., Amruta Borkar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/51055/ > --- > > (Updated Aug. 15, 2016, 11:54 p.m.) > > > Review request for Ambari, Di Li and Robert Nettleton. > > > Bugs: AMBARI-18140 > https://issues.apache.org/jira/browse/AMBARI-18140 > > > Repository: ambari > > > Description > --- > > If a cluster is created with hardcoded hostnames, exported blueprint > continues to include the hardcoded hostnames for hive and oozie external > database properties. This patch updates the existing maps to included the > missed properties. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java > 849e90a > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java > 13927f8 > > Diff: https://reviews.apache.org/r/51055/diff/ > > > Testing > --- > > Test cases included in the patch. > Manual testing performed by creating cluster using blueprint with > hive_existing_mysql_host property set. and tested export blueprint on it. > hive_existing_mysql_host was filtered out in the exported blueprint as > expected. > > > Thanks, > > Amruta Borkar > >
Review Request 51055: Avoid hardcoded values in blueprint for hive and oozie
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51055/ --- Review request for Ambari, Di Li and Robert Nettleton. Bugs: AMBARI-18140 https://issues.apache.org/jira/browse/AMBARI-18140 Repository: ambari Description --- If a cluster is created with hardcoded hostnames, exported blueprint continues to include the hardcoded hostnames for hive and oozie external database properties. This patch updates the existing maps to included the missed properties. Diffs - ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java 849e90a ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java 13927f8 Diff: https://reviews.apache.org/r/51055/diff/ Testing --- Test cases included in the patch Thanks, Amruta Borkar
Re: Review Request 50870: Introduce validation of hostgroup mapping for active/standby namenode
> On Aug. 9, 2016, 8:07 p.m., Di Li wrote: > > Ship It! Thank you Di! Could you please help me push this to trunk? - Amruta --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/50870/#review145255 --- On Aug. 9, 2016, 5:15 p.m., Amruta Borkar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/50870/ > --- > > (Updated Aug. 9, 2016, 5:15 p.m.) > > > Review request for Ambari, Di Li and Robert Nettleton. > > > Bugs: AMBARI-17951 > https://issues.apache.org/jira/browse/AMBARI-17951 > > > Repository: ambari > > > Description > --- > > Ambari allows user to specify following properties when Namenode HA is > enabled, but does not validate for correct mapping. > > "dfs_ha_initial_namenode_active" : "hostname or hostgroup reference", > "dfs_ha_initial_namenode_standby" : "hostname or hostgroup reference” > > If the values for above properties are mentioned incorrectly following > exception is thrown and NAMENODE fails to start. > > WARN namenode.FSNamesystem (FSNamesystem.java:loadFromDisk(692)) - > Encountered exception loading fsimage > java.io.IOException: NameNode is not formatted. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java > 4776dc6 > > ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintValidatorImpl.java > 45a8c5c > > ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterTopologyImpl.java > 699b82a > > ambari-server/src/main/java/org/apache/ambari/server/topology/HostGroup.java > c0aec68 > > ambari-server/src/main/java/org/apache/ambari/server/topology/HostGroupImpl.java > ddbe0b3 > > ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintImplTest.java > 0608697 > > ambari-server/src/test/java/org/apache/ambari/server/topology/ClusterTopologyImplTest.java > 95e58ae > > Diff: https://reviews.apache.org/r/50870/diff/ > > > Testing > --- > > Included test cases in the patch. > Testing done for the following validation cases: > > 1. Extra NAMENODE component > 2. Invalid HOSTGROUP referenced by a property. > 3. Invalid total number of nodes for NAMENODE HA > 4. Invalid hostname mapping > > > Thanks, > > Amruta Borkar > >
Re: Review Request 50870: Introduce validation of hostgroup mapping for active/standby namenode
> On Aug. 9, 2016, 2:17 p.m., Di Li wrote: > > ambari-server/src/main/java/org/apache/ambari/server/topology/HostGroupImpl.java, > > line 288 > > <https://reviews.apache.org/r/50870/diff/2/?file=1466423#file1466423line288> > > > > what's the purpose of the tailing whitespace ? removed the tailing space, as it is not required - Amruta --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/50870/#review145215 ------- On Aug. 9, 2016, 5:15 p.m., Amruta Borkar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/50870/ > --- > > (Updated Aug. 9, 2016, 5:15 p.m.) > > > Review request for Ambari, Di Li and Robert Nettleton. > > > Bugs: AMBARI-17951 > https://issues.apache.org/jira/browse/AMBARI-17951 > > > Repository: ambari > > > Description > --- > > Ambari allows user to specify following properties when Namenode HA is > enabled, but does not validate for correct mapping. > > "dfs_ha_initial_namenode_active" : "hostname or hostgroup reference", > "dfs_ha_initial_namenode_standby" : "hostname or hostgroup reference” > > If the values for above properties are mentioned incorrectly following > exception is thrown and NAMENODE fails to start. > > WARN namenode.FSNamesystem (FSNamesystem.java:loadFromDisk(692)) - > Encountered exception loading fsimage > java.io.IOException: NameNode is not formatted. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java > 4776dc6 > > ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintValidatorImpl.java > 45a8c5c > > ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterTopologyImpl.java > 699b82a > > ambari-server/src/main/java/org/apache/ambari/server/topology/HostGroup.java > c0aec68 > > ambari-server/src/main/java/org/apache/ambari/server/topology/HostGroupImpl.java > ddbe0b3 > > ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintImplTest.java > 0608697 > > ambari-server/src/test/java/org/apache/ambari/server/topology/ClusterTopologyImplTest.java > 95e58ae > > Diff: https://reviews.apache.org/r/50870/diff/ > > > Testing > --- > > Included test cases in the patch. > Testing done for the following validation cases: > > 1. Extra NAMENODE component > 2. Invalid HOSTGROUP referenced by a property. > 3. Invalid total number of nodes for NAMENODE HA > 4. Invalid hostname mapping > > > Thanks, > > Amruta Borkar > >
Re: Review Request 50870: Introduce validation of hostgroup mapping for active/standby namenode
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/50870/ --- (Updated Aug. 9, 2016, 5:15 p.m.) Review request for Ambari, Di Li and Robert Nettleton. Bugs: AMBARI-17951 https://issues.apache.org/jira/browse/AMBARI-17951 Repository: ambari Description --- Ambari allows user to specify following properties when Namenode HA is enabled, but does not validate for correct mapping. "dfs_ha_initial_namenode_active" : "hostname or hostgroup reference", "dfs_ha_initial_namenode_standby" : "hostname or hostgroup reference” If the values for above properties are mentioned incorrectly following exception is thrown and NAMENODE fails to start. WARN namenode.FSNamesystem (FSNamesystem.java:loadFromDisk(692)) - Encountered exception loading fsimage java.io.IOException: NameNode is not formatted. Diffs (updated) - ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java 4776dc6 ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintValidatorImpl.java 45a8c5c ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterTopologyImpl.java 699b82a ambari-server/src/main/java/org/apache/ambari/server/topology/HostGroup.java c0aec68 ambari-server/src/main/java/org/apache/ambari/server/topology/HostGroupImpl.java ddbe0b3 ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintImplTest.java 0608697 ambari-server/src/test/java/org/apache/ambari/server/topology/ClusterTopologyImplTest.java 95e58ae Diff: https://reviews.apache.org/r/50870/diff/ Testing --- Included test cases in the patch. Testing done for the following validation cases: 1. Extra NAMENODE component 2. Invalid HOSTGROUP referenced by a property. 3. Invalid total number of nodes for NAMENODE HA 4. Invalid hostname mapping Thanks, Amruta Borkar
Review Request 50870: Introduce validation of hostgroup mapping for active/standby namenode
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/50870/ --- Review request for Ambari, Balázs Bence Sári and Robert Nettleton. Bugs: AMBARI-17951 https://issues.apache.org/jira/browse/AMBARI-17951 Repository: ambari Description --- Ambari allows user to specify following properties when Namenode HA is enabled, but does not validate for correct mapping. "dfs_ha_initial_namenode_active" : "hostname or hostgroup reference", "dfs_ha_initial_namenode_standby" : "hostname or hostgroup reference” If the values for above properties are mentioned incorrectly following exception is thrown and NAMENODE fails to start. WARN namenode.FSNamesystem (FSNamesystem.java:loadFromDisk(692)) - Encountered exception loading fsimage java.io.IOException: NameNode is not formatted. Diffs - ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintValidatorImpl.java 45a8c5c ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterTopologyImpl.java 699b82a ambari-server/src/main/java/org/apache/ambari/server/topology/HostGroupImpl.java ddbe0b3 ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintImplTest.java 0608697 ambari-server/src/test/java/org/apache/ambari/server/topology/ClusterTopologyImplTest.java 95e58ae Diff: https://reviews.apache.org/r/50870/diff/ Testing --- Included test cases in the patch. Testing done for the following validation cases: 1. Extra NAMENODE component 2. Invalid HOSTGROUP referenced by a property. 3. Invalid total number of nodes for NAMENODE HA 4. Invalid hostname mapping Thanks, Amruta Borkar
Re: Review Request 45507: Enhance blueprint support for using references
> On April 19, 2016, 8:47 p.m., Robert Nettleton wrote: > > ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintValidatorImpl.java, > > line 87 > > <https://reviews.apache.org/r/45507/diff/1/?file=1320101#file1320101line87> > > > > If the default top-level "default_password" property is not set, I > > believe this validation code would still be useful, so I'm inclined to > > think that this block should stay, perhaps in some modified form. I have moved this method in another class ClusterTopologyImpl.java so that check will be performed to see any references left in the blueprint after default_password substitution is performed. > On April 19, 2016, 8:47 p.m., Robert Nettleton wrote: > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java, > > line 158 > > <https://reviews.apache.org/r/45507/diff/1/?file=1320100#file1320100line158> > > > > I'm a little concerned about removing this filter. > > > > Many times in the past, we've run into situations where a component > > that has just been integrated with Ambari will have password properties > > that are not marked as such in the Stack metadata. > > > > This can end up causing a given password to show up in clear text in an > > exported Blueprint. > > > > It does make sense that this filter would need to be modified for a > > change like the one proposed here, but I think it should be updated, rather > > than removed, to still catch the cases it was intended for (password > > properties that are not annotated as passwords in the stacks). > > Amruta Borkar wrote: > Please refer AMBARI-15406-v1 Reintegrated Password filter and added logic to treat password properties differently. - Amruta --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45507/#review129615 --- On May 12, 2016, 9:40 p.m., Amruta Borkar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45507/ > --- > > (Updated May 12, 2016, 9:40 p.m.) > > > Review request for Ambari, Jayush Luniya, Robert Nettleton, and Yusaku Sako. > > > Bugs: AMBARI-15395 and AMBARI-15406 > https://issues.apache.org/jira/browse/AMBARI-15395 > https://issues.apache.org/jira/browse/AMBARI-15406 > > > Repository: ambari > > > Description > --- > > Enhance blueprints to export placeholder/token for password properties. This > is to avoid user from having tochange the password once the cluster is formed > if a user wishes to do so. > Secret References acting as tokens to password properties would be replaced > by user with appropriate passwords, > If any Secret references are found during cluster deployment with blueprint, > those will be replaced by default_password provided in blueprint template. > Need more comments to make this feature more portable. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java > 9cc7b13 > > ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintValidatorImpl.java > 432c6f8 > > ambari-server/src/main/java/org/apache/ambari/server/topology/RequiredPasswordValidator.java > 98eaa40 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java > 14a718d > > ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintImplTest.java > 0b06eb8 > > ambari-server/src/test/java/org/apache/ambari/server/topology/RequiredPasswordValidatorTest.java > e8a2ff5 > > Diff: https://reviews.apache.org/r/45507/diff/ > > > Testing > --- > > Tested the patch by trying out blueprint export and creating a cluster from > the exported blueprint. Result was: the password tokens were replaced by > default password and cluster was created successfully. > > > File Attachments > > > AMBARI-15406.patch > > https://reviews.apache.org/media/uploaded/files/2016/03/30/8daa09ee-82e5-4cd8-98d3-b1c00f1269b7__AMBARI-15406.patch > AMBARI-15406-v1 > > https://reviews.apache.org/media/uploaded/files/2016/05/12/7faf76cc-a42d-4017-b203-8db19448b09d__AMBARI-15406.patch > > > Thanks, > > Amruta Borkar > >
Re: Review Request 45507: Enhance blueprint support for using references
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45507/ --- (Updated May 12, 2016, 9:40 p.m.) Review request for Ambari, Jayush Luniya, Robert Nettleton, and Yusaku Sako. Changes --- Hi Bob, Jayush, I have addressed the issues marked below in this patch. Please review and provide your comments. Thank you, Amruta Bugs: AMBARI-15395 and AMBARI-15406 https://issues.apache.org/jira/browse/AMBARI-15395 https://issues.apache.org/jira/browse/AMBARI-15406 Repository: ambari Description --- Enhance blueprints to export placeholder/token for password properties. This is to avoid user from having tochange the password once the cluster is formed if a user wishes to do so. Secret References acting as tokens to password properties would be replaced by user with appropriate passwords, If any Secret references are found during cluster deployment with blueprint, those will be replaced by default_password provided in blueprint template. Need more comments to make this feature more portable. Diffs - ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java 9cc7b13 ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintValidatorImpl.java 432c6f8 ambari-server/src/main/java/org/apache/ambari/server/topology/RequiredPasswordValidator.java 98eaa40 ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java 14a718d ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintImplTest.java 0b06eb8 ambari-server/src/test/java/org/apache/ambari/server/topology/RequiredPasswordValidatorTest.java e8a2ff5 Diff: https://reviews.apache.org/r/45507/diff/ Testing --- Tested the patch by trying out blueprint export and creating a cluster from the exported blueprint. Result was: the password tokens were replaced by default password and cluster was created successfully. File Attachments (updated) AMBARI-15406.patch https://reviews.apache.org/media/uploaded/files/2016/03/30/8daa09ee-82e5-4cd8-98d3-b1c00f1269b7__AMBARI-15406.patch AMBARI-15406-v1 https://reviews.apache.org/media/uploaded/files/2016/05/12/7faf76cc-a42d-4017-b203-8db19448b09d__AMBARI-15406.patch Thanks, Amruta Borkar
Re: Review Request 45507: Enhance blueprint support for using references
> On April 19, 2016, 8:47 p.m., Robert Nettleton wrote: > > I've taken a quick look at this, and I have a few concerns with this patch: > > > > 1. I'm not sure there is that much utility in replacing the secret tokens > > with the "default_password" value specified in the Cluster Creation > > Template. It should be noted that the "default_password" was never meant > > as a production-level feature, and is there mainly for the convenience of > > developers building new Blueprints. With that in mind, I'm not sure it > > makes sense to introduce this kind of change. In most production scenarios, > > it is likely that most of the passwords will differ, and will need to be > > handled manually anyway. In those cases, it is usually best to configure > > the passwords in a Cluster Creation Template, keeping the Blueprint as > > portable as possible. > > 2. I've mentioned this in my comments, but I'm a little concerned about > > removing the Password property filter, for the reasons I list below. We've > > hit many situations in which a new set of stack configurations cause a > > "password" to show up in clear text, and this filter is very useful in > > keeping much of that from being a problem. If a change like the one > > proposed goes through, at the very least this filter should be > > re-introduced, and modified to account for any changes made. > > > > Thanks. Hello Bob, Thank you for your feedback, we are working on a document that would explain this and relevant functionality in more details. I also agree with some of the issues listed here and I am working on it. Thanks - Amruta --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45507/#review129615 --- On March 30, 2016, 10:19 p.m., Amruta Borkar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45507/ > --- > > (Updated March 30, 2016, 10:19 p.m.) > > > Review request for Ambari and Robert Nettleton. > > > Bugs: AMBARI-15395 and AMBARI-15406 > https://issues.apache.org/jira/browse/AMBARI-15395 > https://issues.apache.org/jira/browse/AMBARI-15406 > > > Repository: ambari > > > Description > --- > > Enhance blueprints to export placeholder/token for password properties. This > is to avoid user from having tochange the password once the cluster is formed > if a user wishes to do so. > Secret References acting as tokens to password properties would be replaced > by user with appropriate passwords, > If any Secret references are found during cluster deployment with blueprint, > those will be replaced by default_password provided in blueprint template. > Need more comments to make this feature more portable. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java > 9cc7b13 > > ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintValidatorImpl.java > 432c6f8 > > ambari-server/src/main/java/org/apache/ambari/server/topology/RequiredPasswordValidator.java > 98eaa40 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java > 14a718d > > ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintImplTest.java > 0b06eb8 > > ambari-server/src/test/java/org/apache/ambari/server/topology/RequiredPasswordValidatorTest.java > e8a2ff5 > > Diff: https://reviews.apache.org/r/45507/diff/ > > > Testing > --- > > Tested the patch by trying out blueprint export and creating a cluster from > the exported blueprint. Result was: the password tokens were replaced by > default password and cluster was created successfully. > > > File Attachments > > > AMBARI-15406.patch > > https://reviews.apache.org/media/uploaded/files/2016/03/30/8daa09ee-82e5-4cd8-98d3-b1c00f1269b7__AMBARI-15406.patch > > > Thanks, > > Amruta Borkar > >
Review Request 45507: Enhance blueprint support for using references
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45507/ --- Review request for Ambari and Robert Nettleton. Bugs: AMBARI-15395 and AMBARI-15406 https://issues.apache.org/jira/browse/AMBARI-15395 https://issues.apache.org/jira/browse/AMBARI-15406 Repository: ambari Description --- Enhance blueprints to export placeholder/token for password properties. This is to avoid user from having tochange the password once the cluster is formed if a user wishes to do so. Secret References acting as tokens to password properties would be replaced by user with appropriate passwords, If any Secret references are found during cluster deployment with blueprint, those will be replaced by default_password provided in blueprint template. Need more comments to make this feature more portable. Diffs - ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java 9cc7b13 ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintValidatorImpl.java 432c6f8 ambari-server/src/main/java/org/apache/ambari/server/topology/RequiredPasswordValidator.java 98eaa40 ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java 14a718d ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintImplTest.java 0b06eb8 ambari-server/src/test/java/org/apache/ambari/server/topology/RequiredPasswordValidatorTest.java e8a2ff5 Diff: https://reviews.apache.org/r/45507/diff/ Testing --- Tested the patch by trying out blueprint export and creating a cluster from the exported blueprint. Result was: the password tokens were replaced by default password and cluster was created successfully. File Attachments AMBARI-15406.patch https://reviews.apache.org/media/uploaded/files/2016/03/30/8daa09ee-82e5-4cd8-98d3-b1c00f1269b7__AMBARI-15406.patch Thanks, Amruta Borkar
Re: Review Request 44725: After exporting blueprint from ranger enabled cluster ranger.service.https.attrib.keystore.pass is exported
> On March 23, 2016, 12:31 p.m., Robert Levas wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java,
> > line 2723
> > <https://reviews.apache.org/r/44725/diff/4/?file=1310989#file1310989line2723>
> >
> > This is a great idea. I am suprised we haven't done this already.
> > However, I don't see where this new filter class is being used, so how was
> > the issue in the description addressed?
>
> Robert Nettleton wrote:
> Hi Rob, the Blueprint export filters have been around for a while, check
> out:
>
>
> org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessor#shouldPropertyBeExcludedForBlueprintExport
>
> Basically, this method iterates over the registered filters to determine
> if a property should be excluded.
>
> Thanks.
>
> Robert Levas wrote:
> Thnaks for the clarification Bob. Droppng my issue.
>
> Amruta Borkar wrote:
> Thank you for the review. I don't have the commit rights on trunk. Could
> you please help push this change?
>
> Robert Levas wrote:
> I'll commit this.
Thank you Robert, could you please update when committed. Thanks.
- Amruta
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44725/#review125010
---
On March 23, 2016, 1:40 a.m., Amruta Borkar wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44725/
> ---
>
> (Updated March 23, 2016, 1:40 a.m.)
>
>
> Review request for Ambari, Robert Levas and Robert Nettleton.
>
>
> Bugs: AMBARI-15338
> https://issues.apache.org/jira/browse/AMBARI-15338
>
>
> Repository: ambari
>
>
> Description
> ---
>
> After exporting blueprint from ranger enabled cluster
> ranger.service.https.attrib.keystore.pass is exported.
> Which needs to be removed before using the same blueprint to create another
> cluster
> Error Show when used same blueprint:
> { "status" : 400, "message" : "Blueprint configuration validation failed:
> Secret references are not allowed in blueprints, replace following properties
> with real passwords:\n Config:ranger-admin-site
> Property:ranger.service.https.attrib.keystore.pass\n" }
>
>
> Diffs
> -
>
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
> 4230862
>
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
> 0f62b2c
>
> Diff: https://reviews.apache.org/r/44725/diff/
>
>
> Testing
> ---
>
> Modified test cases to test for if the properties that end with "pass" are
> getting filtered. Other properties which have 'pass' else where in the name
> will not get filtered.
>
>
> Thanks,
>
> Amruta Borkar
>
>
Re: Review Request 44972: Improve error logging for install errors during blueprint deployments.
> On March 24, 2016, 4:24 p.m., Robert Levas wrote: > > What branch is this for - trunk and branch-2.2? There are two different patches for trunk and branch-2.2 as the code has been refactored in trunk. I do not have commit righs on trunk, I request you to push it to trunk once it gets 'ship it' from Alejandro. - Amruta --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/44972/#review125275 --- On March 23, 2016, 10:59 p.m., Amruta Borkar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/44972/ > --- > > (Updated March 23, 2016, 10:59 p.m.) > > > Review request for Ambari, Alejandro Fernandez and Robert Levas. > > > Bugs: AMBARI-15412 > https://issues.apache.org/jira/browse/AMBARI-15412 > > > Repository: ambari > > > Description > --- > > Improve error logging for install errors during blueprint deployments. > > Currently a severe error during install of a service component gets logged as > a WARNing > > E.g.: > > 09 Mar 2016 12:11:45,881 WARN [qtp-ambari-agent-146] HeartBeatHandler:603 - > Operation failed - may be retried. Service component host: KAFKA_BROKER, > host: hdtest159.svl.ibm.com Action id12-0 > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java > 24fea22 > > Diff: https://reviews.apache.org/r/44972/diff/ > > > Testing > --- > > There are no JUnit test cases, but attached the screenshot which shows > modified output. > > > File Attachments > > > Output > > https://reviews.apache.org/media/uploaded/files/2016/03/17/923d218f-7ca4-4439-b42f-743511936f94__AMBARI-15412_output.png > AMBARI-15412_branch-2.2.patch > > https://reviews.apache.org/media/uploaded/files/2016/03/23/80355c33-2d5e-45f0-8f3d-1640f4386f05__AMBARI-15412_branch-2.2.patch > trunk patch > > https://reviews.apache.org/media/uploaded/files/2016/03/23/5c7a7d2f-fcf7-4858-9626-6a08ec9c7ba7__AMBARI-15412-trunk.patch > trunk--output > > https://reviews.apache.org/media/uploaded/files/2016/03/23/41b8327f-40ae-4a2f-b9c3-4e85eb9004c4__trunk-output.png > > > Thanks, > > Amruta Borkar > >
Re: Review Request 44972: Improve error logging for install errors during blueprint deployments.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/44972/ --- (Updated March 23, 2016, 10:54 p.m.) Review request for Ambari, Di Li and Sid Wagle. Changes --- Made siilar changes to address the issue in trunk. Trunk test output is attached. Bugs: AMBARI-15412 https://issues.apache.org/jira/browse/AMBARI-15412 Repository: ambari Description --- Improve error logging for install errors during blueprint deployments. Currently a severe error during install of a service component gets logged as a WARNing E.g.: 09 Mar 2016 12:11:45,881 WARN [qtp-ambari-agent-146] HeartBeatHandler:603 - Operation failed - may be retried. Service component host: KAFKA_BROKER, host: hdtest159.svl.ibm.com Action id12-0 Diffs - ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java 24fea22 Diff: https://reviews.apache.org/r/44972/diff/ Testing --- There are no JUnit test cases, but attached the screenshot which shows modified output. File Attachments (updated) Output https://reviews.apache.org/media/uploaded/files/2016/03/17/923d218f-7ca4-4439-b42f-743511936f94__AMBARI-15412_output.png AMBARI-15412_branch-2.2.patch https://reviews.apache.org/media/uploaded/files/2016/03/23/80355c33-2d5e-45f0-8f3d-1640f4386f05__AMBARI-15412_branch-2.2.patch trunk patch https://reviews.apache.org/media/uploaded/files/2016/03/23/5c7a7d2f-fcf7-4858-9626-6a08ec9c7ba7__AMBARI-15412-trunk.patch trunk--output https://reviews.apache.org/media/uploaded/files/2016/03/23/41b8327f-40ae-4a2f-b9c3-4e85eb9004c4__trunk-output.png Thanks, Amruta Borkar
Re: Review Request 44725: After exporting blueprint from ranger enabled cluster ranger.service.https.attrib.keystore.pass is exported
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44725/
---
(Updated March 22, 2016, 6:41 p.m.)
Review request for Ambari, Di Li and Robert Nettleton.
Changes
---
Uploading new patch with suggested changes. Created a new filter class to
filter properties based on property_type present in Stack. Added test cases for
the same.
Bugs: AMBARI-15338
https://issues.apache.org/jira/browse/AMBARI-15338
Repository: ambari
Description
---
After exporting blueprint from ranger enabled cluster
ranger.service.https.attrib.keystore.pass is exported.
Which needs to be removed before using the same blueprint to create another
cluster
Error Show when used same blueprint:
{ "status" : 400, "message" : "Blueprint configuration validation failed:
Secret references are not allowed in blueprints, replace following properties
with real passwords:\n Config:ranger-admin-site
Property:ranger.service.https.attrib.keystore.pass\n" }
Diffs (updated)
-
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
4230862
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
0f62b2c
Diff: https://reviews.apache.org/r/44725/diff/
Testing
---
Modified test cases to test for if the properties that end with "pass" are
getting filtered. Other properties which have 'pass' else where in the name
will not get filtered.
Thanks,
Amruta Borkar
Re: Review Request 44972: Improve error logging for install errors during blueprint deployments.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/44972/ --- (Updated March 17, 2016, 6:11 p.m.) Review request for Ambari, Di Li and Sumit Mohanty. Bugs: AMBARI-15412 https://issues.apache.org/jira/browse/AMBARI-15412 Repository: ambari Description (updated) --- Improve error logging for install errors during blueprint deployments. Currently a severe error during install of a service component gets logged as a WARNing E.g.: 09 Mar 2016 12:11:45,881 WARN [qtp-ambari-agent-146] HeartBeatHandler:603 - Operation failed - may be retried. Service component host: KAFKA_BROKER, host: hdtest159.svl.ibm.com Action id12-0 Diffs - ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java 24fea22 Diff: https://reviews.apache.org/r/44972/diff/ Testing --- There are no JUnit test cases, but attached the screenshot which shows modified output. File Attachments Patch https://reviews.apache.org/media/uploaded/files/2016/03/17/2110d517-0cf4-4477-87e7-935c963e04de__AMBARI-15412.patch Output https://reviews.apache.org/media/uploaded/files/2016/03/17/923d218f-7ca4-4439-b42f-743511936f94__AMBARI-15412_output.png Thanks, Amruta Borkar
Re: Review Request 44725: After exporting blueprint from ranger enabled cluster ranger.service.https.attrib.keystore.pass is exported
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44725/
---
(Updated March 17, 2016, 6:52 p.m.)
Review request for Ambari, Di Li and Robert Nettleton.
Changes
---
Currently Secret References are getting filtered based on Property Name, but as
properties are ending with difference strings (password, secret, pass), the
filtering expression needs to be modified every time.
As these password properties have values that follow the same format
"SECRET:property_name:version:property_name" we can filter Secret References
based on property value, rather than property name, as this would make it
generic.
Bugs: AMBARI-15338
https://issues.apache.org/jira/browse/AMBARI-15338
Repository: ambari
Description
---
After exporting blueprint from ranger enabled cluster
ranger.service.https.attrib.keystore.pass is exported.
Which needs to be removed before using the same blueprint to create another
cluster
Error Show when used same blueprint:
{ "status" : 400, "message" : "Blueprint configuration validation failed:
Secret references are not allowed in blueprints, replace following properties
with real passwords:\n Config:ranger-admin-site
Property:ranger.service.https.attrib.keystore.pass\n" }
Diffs (updated)
-
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
f5e7578
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
68d5755
Diff: https://reviews.apache.org/r/44725/diff/
Testing
---
Modified test cases to test for if the properties that end with "pass" are
getting filtered. Other properties which have 'pass' else where in the name
will not get filtered.
Thanks,
Amruta Borkar
Re: Review Request 44725: After exporting blueprint from ranger enabled cluster ranger.service.https.attrib.keystore.pass is exported
> On March 14, 2016, 12:27 p.m., Di Li wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java,
> > line 2668
> > <https://reviews.apache.org/r/44725/diff/2/?file=1296341#file1296341line2668>
> >
> > So any properties end with ".pass" will be marked, yes? That check
> > seems a bit broad to me, at least broader than checking Password or Secret.
> >
> > Each property (understandably that only when its defintion follows the
> > rules) should use "property-type" to indicate if it's a password
> > "PASSWORD". Can we check this instead of
> > guessing property name patterns?
> >
> > We should honor the setting, so that when user needs to mask a
> > property, he can either ends the property name with .password or .secret
> > (this is really a hardcoded logic than stack driven) or configure the
> > property propertly (stack driven, a more preferrable way).
Hello Di,
Property_type is not accessible to the DAO part. It shows only at UI. So I have
tried to generalise the expression using Property_value rather than
property_name. Please give your input for that.
Thanks
- Amruta
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44725/#review123409
---
On March 17, 2016, 6:52 p.m., Amruta Borkar wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44725/
> ---
>
> (Updated March 17, 2016, 6:52 p.m.)
>
>
> Review request for Ambari, Di Li and Robert Nettleton.
>
>
> Bugs: AMBARI-15338
> https://issues.apache.org/jira/browse/AMBARI-15338
>
>
> Repository: ambari
>
>
> Description
> ---
>
> After exporting blueprint from ranger enabled cluster
> ranger.service.https.attrib.keystore.pass is exported.
> Which needs to be removed before using the same blueprint to create another
> cluster
> Error Show when used same blueprint:
> { "status" : 400, "message" : "Blueprint configuration validation failed:
> Secret references are not allowed in blueprints, replace following properties
> with real passwords:\n Config:ranger-admin-site
> Property:ranger.service.https.attrib.keystore.pass\n" }
>
>
> Diffs
> -
>
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
> f5e7578
>
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
> 68d5755
>
> Diff: https://reviews.apache.org/r/44725/diff/
>
>
> Testing
> ---
>
> Modified test cases to test for if the properties that end with "pass" are
> getting filtered. Other properties which have 'pass' else where in the name
> will not get filtered.
>
>
> Thanks,
>
> Amruta Borkar
>
>
Review Request 44972: Improve error logging for install errors during blueprint deployments.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/44972/ --- Review request for Ambari, Di Li and Sumit Mohanty. Bugs: AMBARI-15412 https://issues.apache.org/jira/browse/AMBARI-15412 Repository: ambari Description --- Improve error logging for install errors during blueprint deployments. Currently a severe error during install of a service component gets logged as a WARNing. E.g.: 09 Mar 2016 12:11:45,881 WARN [qtp-ambari-agent-146] HeartBeatHandler:603 - Operation failed - may be retried. Service component host: KAFKA_BROKER, host: hdtest159.svl.ibm.com Action id12-0 Diffs - ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java 24fea22 Diff: https://reviews.apache.org/r/44972/diff/ Testing --- There are no JUnit test cases, but attached the screenshot which shows modified output. File Attachments Patch https://reviews.apache.org/media/uploaded/files/2016/03/17/2110d517-0cf4-4477-87e7-935c963e04de__AMBARI-15412.patch Output https://reviews.apache.org/media/uploaded/files/2016/03/17/923d218f-7ca4-4439-b42f-743511936f94__AMBARI-15412_output.png Thanks, Amruta Borkar
