Re: Review Request 65963: AMS restart failed in embedded mode
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/65963/#review198880 --- We no longer use Review Board for code reviews. See https://cwiki.apache.org/confluence/display/AMBARI/How+to+Contribute for the new process. - Nate Cole On March 7, 2018, 10:26 p.m., wang jianfei wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/65963/ > --- > > (Updated March 7, 2018, 10:26 p.m.) > > > Review request for Ambari and Aravindan Vijayan. > > > Bugs: AMBARI-23178 > https://issues.apache.org/jira/browse/AMBARI-23178 > > > Repository: ambari > > > Description > --- > > While ams use embedded mode,you can start it for the first time.But,when you > restart it,it starts failed.The log of hbase master is shown above.The reason > is due to the existed hbase.zookeeper.property.dataDir before.So,we must > delete it everytime before you want to restart ams in embedded mode. > > > Diffs > - > > > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/ams_service.py > 875211f > > > Diff: https://reviews.apache.org/r/65963/diff/1/ > > > Testing > --- > > Manually tested > > > Thanks, > > wang jianfei > >
Re: Review Request 64769: AMBARI-22406 : UI for verifying if repo exist for new OS family and user prompt for entering HDP and HDP-UTILs repo URLs
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64769/#review194955 --- This has been pushed. Please close this review and associated jira. - Nate Cole On Jan. 8, 2018, 8:05 a.m., Sonia Garudi wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64769/ > --- > > (Updated Jan. 8, 2018, 8:05 a.m.) > > > Review request for Ambari and Nate Cole. > > > Bugs: AMBARI-22406 > https://issues.apache.org/jira/browse/AMBARI-22406 > > > Repository: ambari > > > Description > --- > > Includes UI changes in ambari-web to check if the repo is selected or > provides prompt to the user to enter the HDP and HDP-utils repo URL's. > Includes necessary validation checks for the repo urls, allows user to select > the public and local repository and also provides functionality to skip > validation checks. > > > Diffs > - > > ambari-web/app/controllers/wizard/step3_controller.js 55f4323 > ambari-web/app/messages.js 6da0368 > ambari-web/app/styles/wizard.less 4e680ca > ambari-web/app/templates/wizard/step3.hbs ff436a8 > ambari-web/app/utils/ajax/ajax.js 483be5e > ambari-web/app/views/wizard/step3/hostWarningPopupBody_view.js 786c4f7 > ambari-web/app/views/wizard/step3_view.js 3249b24 > ambari-web/test/controllers/wizard/step3_test.js b8bc794 > > > Diff: https://reviews.apache.org/r/64769/diff/5/ > > > Testing > --- > > Tests were successful in ambari-web > > > Thanks, > > Sonia Garudi > >
Re: Review Request 64769: AMBARI-22406 : UI for verifying if repo exist for new OS family and user prompt for entering HDP and HDP-UTILs repo URLs
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64769/#review194954 --- Ship it! Ship It! - Nate Cole On Jan. 8, 2018, 8:05 a.m., Sonia Garudi wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64769/ > --- > > (Updated Jan. 8, 2018, 8:05 a.m.) > > > Review request for Ambari and Nate Cole. > > > Bugs: AMBARI-22406 > https://issues.apache.org/jira/browse/AMBARI-22406 > > > Repository: ambari > > > Description > --- > > Includes UI changes in ambari-web to check if the repo is selected or > provides prompt to the user to enter the HDP and HDP-utils repo URL's. > Includes necessary validation checks for the repo urls, allows user to select > the public and local repository and also provides functionality to skip > validation checks. > > > Diffs > - > > ambari-web/app/controllers/wizard/step3_controller.js 55f4323 > ambari-web/app/messages.js 6da0368 > ambari-web/app/styles/wizard.less 4e680ca > ambari-web/app/templates/wizard/step3.hbs ff436a8 > ambari-web/app/utils/ajax/ajax.js 483be5e > ambari-web/app/views/wizard/step3/hostWarningPopupBody_view.js 786c4f7 > ambari-web/app/views/wizard/step3_view.js 3249b24 > ambari-web/test/controllers/wizard/step3_test.js b8bc794 > > > Diff: https://reviews.apache.org/r/64769/diff/5/ > > > Testing > --- > > Tests were successful in ambari-web > > > Thanks, > > Sonia Garudi > >
Re: Review Request 64974: Setup fast-hdfs-resource.jar for OneFS
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64974/#review194869 --- Ship it! Ship It! - Nate Cole On Jan. 5, 2018, 5:50 a.m., Attila Magyar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64974/ > --- > > (Updated Jan. 5, 2018, 5:50 a.m.) > > > Review request for Ambari, Nate Cole and Robert Levas. > > > Bugs: AMBARI-22121 > https://issues.apache.org/jira/browse/AMBARI-22121 > > > Repository: ambari > > > Description > --- > > Normally the shared_initialization python script takes care of creating the > fast-hdfs-resource.jar. This script is executed by a before-START hook. If > you install OneFS alone, no start command will be generated because OneFS is > a client only service. > > > Diffs > - > > > contrib/management-packs/isilon-onefs-mpack/src/main/resources/addon-services/ONEFS/1.0.0/metainfo.xml > 06a90d9 > > contrib/management-packs/isilon-onefs-mpack/src/main/resources/addon-services/ONEFS/1.0.0/package/scripts/onefs_client.py > dbf1331 > > contrib/management-packs/isilon-onefs-mpack/src/main/resources/addon-services/ONEFS/1.0.0/package/scripts/params_linux.py > 79aff8f > > > Diff: https://reviews.apache.org/r/64974/diff/1/ > > > Testing > --- > > - installed OneFS alone > > > Thanks, > > Attila Magyar > >
Re: Review Request 64956: Update Hadoop RPC Encryption Properties During Kerberization and Upgrade
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64956/#review194787
---
Ship it!
Ship It!
- Nate Cole
On Jan. 4, 2018, 3:16 p.m., Jonathan Hurley wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64956/
> ---
>
> (Updated Jan. 4, 2018, 3:16 p.m.)
>
>
> Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, and Nate Cole.
>
>
> Bugs: AMBARI-22725
> https://issues.apache.org/jira/browse/AMBARI-22725
>
>
> Repository: ambari
>
>
> Description
> ---
>
> Clients should have the ability to choose encrypted communication over RPC
> when talking to core hadoop components. Today, the properties that control
> this are:
>
> - {{core-site.xml : hadoop.rpc.protection = authentication}}
> - {{hdfs-site.xml : dfs.data.transfer.protection = authentication}}
>
> The new value of {{privacy}} enables clients to choose an encrypted means of
> communication. By keeping {{authentication}} first, it will be taken as the
> default mechanism so that wire encryption is not automatically enabled by
> accident.
>
> The following properties should be changed to add {{privacy}}:
>
> - {{core-site.xml : hadoop.rpc.protection = authentication,privacy}}
> - {{hdfs-site.xml : dfs.data.transfer.protection = authentication,privacy}}
>
> The following are cases when this needs to be performed:
> - During Kerberization, the above two properties should be automatically
> reconfigured.
> - During a stack upgrade to any version of HDP 2.6, they should be
> automatically merged
>
> Blueprint deployment is not a scenario being covered here.
>
>
> Diffs
> -
>
>
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/ConfigureAction.java
> fbcde51b32
>
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ClusterGrouping.java
> c1a05c03b4
>
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/StageWrapperBuilder.java
> 7fd8938f7d
>
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/Task.java
> 2167b7b464
> ambari-server/src/main/resources/stacks/HDP/2.2/services/stack_advisor.py
> 0b0c9c56af
> ambari-server/src/main/resources/stacks/HDP/2.6/services/HDFS/kerberos.json
> f8bdc5cc5c
> ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/config-upgrade.xml
> 94787225f1
>
> ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/nonrolling-upgrade-2.6.xml
> d506f1f16c
> ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/upgrade-2.6.xml
> 17a63943ba
> ambari-server/src/main/resources/upgrade-pack.xsd 9e50a087b6
>
> ambari-server/src/test/java/org/apache/ambari/server/state/UpgradeHelperTest.java
> ac9be666c7
>
> ambari-server/src/test/resources/stacks/HDP/2.2.0/upgrades/upgrade_test_conditions.xml
> 61c891a4be
>
>
> Diff: https://reviews.apache.org/r/64956/diff/2/
>
>
> Testing
> ---
>
> PENDING
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 64769: AMBARI-22406 : UI for verifying if repo exist for new OS family and user prompt for entering HDP and HDP-UTILs repo URLs
> On Dec. 22, 2017, 9:51 a.m., Nate Cole wrote: > > ambari-web/app/controllers/wizard/step3_controller.js > > Lines 863-880 (patched) > > <https://reviews.apache.org/r/64769/diff/2/?file=1925966#file1925966line863> > > > > I'm no UI expert, but this entire new block seems to be doing repo > > stuff that already is on the UI. Can you explain the gist here? > > Sonia Garudi wrote: > We need each host's os type which is not available on the UI. This > function makes an API call which gets information of all the bootstrapped > hosts . > > Nate Cole wrote: > I see. We should probably have that returned as part of host > registration and read it from there instead of making another call. Is that > possible in this patch? > > Sonia Garudi wrote: > The os types of the bootstrapped hosts can be obtained in the > registration success callback(isHostsRegisteredSuccessCallback) which would > eliminate the above API call. This would involve editing the > 'wizard.step3.is_hosts_registered' API call in ajax.js to include the > 'Hosts/os_type' field. > Please let me know your suggestions. > > Nate Cole wrote: > That sounds reasonable instead of making another call. Can we make that > an error/failure case after registration that we don't have a JAVA_HOME > matching the type? > > Yussuf Shaikh wrote: > Do you want us to fail the Host Registration if java home is not found > for host os_type? > We were planning to just show a message on the host check warnings popup > that Host does not have proper Java Home set of its Os type and default java > home will be used. This assumption is because during normal(existing) flow > Amabari show warning for incorrect Java Home and does not fail registration. No need to fail host registration. The popup informational is fine. - Nate --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64769/#review194434 --- On Jan. 4, 2018, 7:07 a.m., Sonia Garudi wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64769/ > --- > > (Updated Jan. 4, 2018, 7:07 a.m.) > > > Review request for Ambari and Nate Cole. > > > Bugs: AMBARI-22406 > https://issues.apache.org/jira/browse/AMBARI-22406 > > > Repository: ambari > > > Description > --- > > Includes UI changes in ambari-web to check if the repo is selected or > provides prompt to the user to enter the HDP and HDP-utils repo URL's. > Includes necessary validation checks for the repo urls, allows user to select > the public and local repository and also provides functionality to skip > validation checks. > > > Diffs > - > > ambari-web/app/controllers/wizard/step3_controller.js 55f4323 > ambari-web/app/styles/wizard.less 4e680ca > ambari-web/app/templates/wizard/step3.hbs ff436a8 > ambari-web/app/utils/ajax/ajax.js 483be5e > ambari-web/app/views/wizard/step3_view.js 3249b24 > ambari-web/test/controllers/wizard/step3_test.js b8bc794 > > > Diff: https://reviews.apache.org/r/64769/diff/4/ > > > Testing > --- > > Tests were successful in ambari-web > > > Thanks, > > Sonia Garudi > >
Re: Review Request 64769: AMBARI-22406 : UI for verifying if repo exist for new OS family and user prompt for entering HDP and HDP-UTILs repo URLs
> On Dec. 22, 2017, 9:51 a.m., Nate Cole wrote: > > ambari-web/app/controllers/wizard/step3_controller.js > > Lines 863-880 (patched) > > <https://reviews.apache.org/r/64769/diff/2/?file=1925966#file1925966line863> > > > > I'm no UI expert, but this entire new block seems to be doing repo > > stuff that already is on the UI. Can you explain the gist here? > > Sonia Garudi wrote: > We need each host's os type which is not available on the UI. This > function makes an API call which gets information of all the bootstrapped > hosts . > > Nate Cole wrote: > I see. We should probably have that returned as part of host > registration and read it from there instead of making another call. Is that > possible in this patch? > > Sonia Garudi wrote: > The os types of the bootstrapped hosts can be obtained in the > registration success callback(isHostsRegisteredSuccessCallback) which would > eliminate the above API call. This would involve editing the > 'wizard.step3.is_hosts_registered' API call in ajax.js to include the > 'Hosts/os_type' field. > Please let me know your suggestions. That sounds reasonable instead of making another call. Can we make that an error/failure case after registration that we don't have a JAVA_HOME matching the type? - Nate --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64769/#review194434 --- On Jan. 4, 2018, 7:07 a.m., Sonia Garudi wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64769/ > ------- > > (Updated Jan. 4, 2018, 7:07 a.m.) > > > Review request for Ambari and Nate Cole. > > > Bugs: AMBARI-22406 > https://issues.apache.org/jira/browse/AMBARI-22406 > > > Repository: ambari > > > Description > --- > > Includes UI changes in ambari-web to check if the repo is selected or > provides prompt to the user to enter the HDP and HDP-utils repo URL's. > Includes necessary validation checks for the repo urls, allows user to select > the public and local repository and also provides functionality to skip > validation checks. > > > Diffs > - > > ambari-web/app/controllers/wizard/step3_controller.js 55f4323 > ambari-web/app/styles/wizard.less 4e680ca > ambari-web/app/templates/wizard/step3.hbs ff436a8 > ambari-web/app/utils/ajax/ajax.js 483be5e > ambari-web/app/views/wizard/step3_view.js 3249b24 > ambari-web/test/controllers/wizard/step3_test.js b8bc794 > > > Diff: https://reviews.apache.org/r/64769/diff/4/ > > > Testing > --- > > Tests were successful in ambari-web > > > Thanks, > > Sonia Garudi > >
Re: Review Request 64887: AMBARI-22403 Read the JAVA_HOME depending on the OS family during Service install (action & command) and upgrade
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64887/#review194764 --- This has been pushed. Please close this review and jira. - Nate Cole On Jan. 3, 2018, 2:03 a.m., Yussuf Shaikh wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64887/ > --- > > (Updated Jan. 3, 2018, 2:03 a.m.) > > > Review request for Ambari and Nate Cole. > > > Bugs: AMBARI-22403 > https://issues.apache.org/jira/browse/AMBARI-22403 > > > Repository: ambari > > > Description > --- > > Java side changes required to read JAVA HOME value for each host depending on > the OS type eg: java.home.redhat7. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java > 182cd58 > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariActionExecutionHelper.java > c0a7a7b > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java > c4df0b1 > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java > 198b617 > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClientConfigResourceProvider.java > dcafdea > ambari-server/src/main/java/org/apache/ambari/server/utils/StageUtils.java > b6287e6 > > ambari-server/src/test/java/org/apache/ambari/server/configuration/ConfigurationTest.java > e47fcd2 > > ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelperTest.java > 6bece66 > > ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerImplTest.java > e795d28 > > ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerTest.java > a1f28f1 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClientConfigResourceProviderTest.java > c5994c5 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProviderTest.java > 51035ba > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/HostStackVersionResourceProviderTest.java > 670c187 > > ambari-server/src/test/java/org/apache/ambari/server/utils/StageUtilsTest.java > b90295d > > > Diff: https://reviews.apache.org/r/64887/diff/2/ > > > Testing > --- > > tests ran on ambari-server project. > > > Thanks, > > Yussuf Shaikh > >
Re: Review Request 64887: AMBARI-22403 Read the JAVA_HOME depending on the OS family during Service install (action & command) and upgrade
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64887/#review194763 --- Ship it! Ship It! - Nate Cole On Jan. 3, 2018, 2:03 a.m., Yussuf Shaikh wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64887/ > --- > > (Updated Jan. 3, 2018, 2:03 a.m.) > > > Review request for Ambari and Nate Cole. > > > Bugs: AMBARI-22403 > https://issues.apache.org/jira/browse/AMBARI-22403 > > > Repository: ambari > > > Description > --- > > Java side changes required to read JAVA HOME value for each host depending on > the OS type eg: java.home.redhat7. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java > 182cd58 > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariActionExecutionHelper.java > c0a7a7b > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java > c4df0b1 > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java > 198b617 > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClientConfigResourceProvider.java > dcafdea > ambari-server/src/main/java/org/apache/ambari/server/utils/StageUtils.java > b6287e6 > > ambari-server/src/test/java/org/apache/ambari/server/configuration/ConfigurationTest.java > e47fcd2 > > ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelperTest.java > 6bece66 > > ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerImplTest.java > e795d28 > > ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerTest.java > a1f28f1 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClientConfigResourceProviderTest.java > c5994c5 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProviderTest.java > 51035ba > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/HostStackVersionResourceProviderTest.java > 670c187 > > ambari-server/src/test/java/org/apache/ambari/server/utils/StageUtilsTest.java > b90295d > > > Diff: https://reviews.apache.org/r/64887/diff/2/ > > > Testing > --- > > tests ran on ambari-server project. > > > Thanks, > > Yussuf Shaikh > >
Re: Review Request 64887: AMBARI-22403 Read the JAVA_HOME depending on the OS family during Service install (action & command) and upgrade
> On Jan. 2, 2018, 11:51 a.m., Nate Cole wrote: > > ambari-server/src/main/java/org/apache/ambari/server/utils/StageUtils.java > > Line 633 (original) > > <https://reviews.apache.org/r/64887/diff/1/?file=1929460#file1929460line633> > > > > It's much more convenient to keep JAVA_HOME in one place. We want to > > centralize (as much as we can) the places where we have to remember to set > > JAVA_HOME. Can you pass in the os type or OS (the object0 here instead? > > Yussuf Shaikh wrote: > Here we do not know the os type of each host. This method is called > initially during stage creation. Hence had introduced a method in > Configuration class which is called in hosts loop later for different flows. Fair enough - Nate --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64887/#review194635 --- On Jan. 3, 2018, 2:03 a.m., Yussuf Shaikh wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64887/ > --- > > (Updated Jan. 3, 2018, 2:03 a.m.) > > > Review request for Ambari and Nate Cole. > > > Bugs: AMBARI-22403 > https://issues.apache.org/jira/browse/AMBARI-22403 > > > Repository: ambari > > > Description > --- > > Java side changes required to read JAVA HOME value for each host depending on > the OS type eg: java.home.redhat7. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java > 182cd58 > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariActionExecutionHelper.java > c0a7a7b > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java > c4df0b1 > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java > 198b617 > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClientConfigResourceProvider.java > dcafdea > ambari-server/src/main/java/org/apache/ambari/server/utils/StageUtils.java > b6287e6 > > ambari-server/src/test/java/org/apache/ambari/server/configuration/ConfigurationTest.java > e47fcd2 > > ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelperTest.java > 6bece66 > > ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerImplTest.java > e795d28 > > ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerTest.java > a1f28f1 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClientConfigResourceProviderTest.java > c5994c5 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProviderTest.java > 51035ba > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/HostStackVersionResourceProviderTest.java > 670c187 > > ambari-server/src/test/java/org/apache/ambari/server/utils/StageUtilsTest.java > b90295d > > > Diff: https://reviews.apache.org/r/64887/diff/2/ > > > Testing > --- > > tests ran on ambari-server project. > > > Thanks, > > Yussuf Shaikh > >
Re: Review Request 64950: Ambari Upgrade failed in case if database have no cluster_version table already. The issue with idempotence
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64950/#review194759
---
Ship it!
Ship It!
- Nate Cole
On Jan. 4, 2018, 9:44 a.m., Dmitro Lisnichenko wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64950/
> ---
>
> (Updated Jan. 4, 2018, 9:44 a.m.)
>
>
> Review request for Ambari, Jonathan Hurley and Nate Cole.
>
>
> Bugs: AMBARI-22723
> https://issues.apache.org/jira/browse/AMBARI-22723
>
>
> Repository: ambari
>
>
> Description
> ---
>
> AMBARI-21728 and AMBARI-22469 make the upgrade broken in case of several
> upgrade tries, failing to follow the Idempotent upgrade paradigm.
>
> Issue is with:
> {code}
> protected void executeDDLUpdates() throws AmbariException, SQLException {
> Integer currentVersionID = getCurrentVersionID();<--
> dropBrokenFK();
> updateServiceComponentDesiredStateTable(currentVersionID);
> <--
> updateServiceDesiredStateTable(currentVersionID); <--
> addSelectedCollumsToClusterconfigTable();
> updateHostComponentDesiredStateTable();
> updateHostComponentStateTable();
> dropStaleTables(); <--
> updateUpgradeTable();
> createUpgradeHistoryTable();
> updateRepositoryVersionTable();
> renameServiceDeletedColumn();
> addLegacyColumn();
> expandUpgradeItemItemTextColumn();
> addViewUrlPKConstraint();
> removeStaleConstraints();
> }
> {code}
>
> {{getCurrentVersionID()}} trying to query non-existing table and failing
> whole upgrade
>
>
> Diffs
> -
>
>
> ambari-server/src/main/java/org/apache/ambari/server/orm/DBAccessorImpl.java
> 391a6f567e
>
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog260.java
> 6ba1671cdf
>
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog260Test.java
> e93cd1303a
>
>
> Diff: https://reviews.apache.org/r/64950/diff/1/
>
>
> Testing
> ---
>
> mvn clean test
> live cluster check
>
>
> Thanks,
>
> Dmitro Lisnichenko
>
>
Re: Review Request 64775: AMBARI-22394: Store and retrieve the ambari repo urls for os family during bootstrap
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64775/#review194638 --- This has been pushed. Please close this review and associated JIRA. - Nate Cole On Dec. 23, 2017, 2:16 a.m., Sneha Kanekar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64775/ > --- > > (Updated Dec. 23, 2017, 2:16 a.m.) > > > Review request for Ambari and Nate Cole. > > > Bugs: AMBARI-22394 > https://issues.apache.org/jira/browse/AMBARI-22394 > > > Repository: ambari > > > Description > --- > > Ambari configuration will have list of repo urls for each os type. During > bootstrap, the stored repo url will be used to correct the base url from > ambari repo file on agent host. > > > Diffs > - > > ambari-server/src/main/python/bootstrap.py f1c53ce > ambari-server/src/main/python/os_check_type.py da37560 > ambari-server/src/main/python/setupAgent.py efc3d7c > ambari-server/src/test/python/TestBootstrap.py bea47f4 > ambari-server/src/test/python/TestSetupAgent.py 0b33e9f > > > Diff: https://reviews.apache.org/r/64775/diff/2/ > > > Testing > --- > > Tests were successful in ambari-server. > > > Thanks, > > Sneha Kanekar > >
Re: Review Request 64775: AMBARI-22394: Store and retrieve the ambari repo urls for os family during bootstrap
> On Dec. 22, 2017, 10:12 a.m., Nate Cole wrote: > > ambari-server/src/main/python/bootstrap.py > > Lines 762-763 (patched) > > <https://reviews.apache.org/r/64775/diff/1/?file=1926153#file1926153line762> > > > > Use "".format() syntax here. > > > > Should we give more information here? How will we tell the user what > > to do? How do they get the os_type into ambari.properties? Please mark these as Fixed when they are updated in a diff. - Nate --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64775/#review194436 --- On Dec. 23, 2017, 2:16 a.m., Sneha Kanekar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64775/ > --- > > (Updated Dec. 23, 2017, 2:16 a.m.) > > > Review request for Ambari and Nate Cole. > > > Bugs: AMBARI-22394 > https://issues.apache.org/jira/browse/AMBARI-22394 > > > Repository: ambari > > > Description > --- > > Ambari configuration will have list of repo urls for each os type. During > bootstrap, the stored repo url will be used to correct the base url from > ambari repo file on agent host. > > > Diffs > - > > ambari-server/src/main/python/bootstrap.py f1c53ce > ambari-server/src/main/python/os_check_type.py da37560 > ambari-server/src/main/python/setupAgent.py efc3d7c > ambari-server/src/test/python/TestBootstrap.py bea47f4 > ambari-server/src/test/python/TestSetupAgent.py 0b33e9f > > > Diff: https://reviews.apache.org/r/64775/diff/2/ > > > Testing > --- > > Tests were successful in ambari-server. > > > Thanks, > > Sneha Kanekar > >
Re: Review Request 64775: AMBARI-22394: Store and retrieve the ambari repo urls for os family during bootstrap
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64775/#review194636 --- Ship it! Ship It! - Nate Cole On Dec. 23, 2017, 2:16 a.m., Sneha Kanekar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64775/ > --- > > (Updated Dec. 23, 2017, 2:16 a.m.) > > > Review request for Ambari and Nate Cole. > > > Bugs: AMBARI-22394 > https://issues.apache.org/jira/browse/AMBARI-22394 > > > Repository: ambari > > > Description > --- > > Ambari configuration will have list of repo urls for each os type. During > bootstrap, the stored repo url will be used to correct the base url from > ambari repo file on agent host. > > > Diffs > - > > ambari-server/src/main/python/bootstrap.py f1c53ce > ambari-server/src/main/python/os_check_type.py da37560 > ambari-server/src/main/python/setupAgent.py efc3d7c > ambari-server/src/test/python/TestBootstrap.py bea47f4 > ambari-server/src/test/python/TestSetupAgent.py 0b33e9f > > > Diff: https://reviews.apache.org/r/64775/diff/2/ > > > Testing > --- > > Tests were successful in ambari-server. > > > Thanks, > > Sneha Kanekar > >
Re: Review Request 64887: AMBARI-22403 Read the JAVA_HOME depending on the OS family during Service install (action & command) and upgrade
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64887/#review194635 --- ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java Lines 3058-3065 (original), 3057-3060 (patched) <https://reviews.apache.org/r/64887/#comment273516> The old logic here is still sound, it would help for when people add properties without having to restart Ambari to get them (not changed properties, but new ones which is fine). ambari-server/src/main/java/org/apache/ambari/server/utils/StageUtils.java Line 633 (original) <https://reviews.apache.org/r/64887/#comment273514> It's much more convenient to keep JAVA_HOME in one place. We want to centralize (as much as we can) the places where we have to remember to set JAVA_HOME. Can you pass in the os type or OS (the object0 here instead? - Nate Cole On Jan. 2, 2018, 9:54 a.m., Yussuf Shaikh wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64887/ > --- > > (Updated Jan. 2, 2018, 9:54 a.m.) > > > Review request for Ambari and Nate Cole. > > > Bugs: AMBARI-22403 > https://issues.apache.org/jira/browse/AMBARI-22403 > > > Repository: ambari > > > Description > --- > > Java side changes required to read JAVA HOME value for each host depending on > the OS type eg: java.home.redhat7. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java > 182cd58 > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariActionExecutionHelper.java > c0a7a7b > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java > c4df0b1 > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java > 198b617 > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClientConfigResourceProvider.java > dcafdea > ambari-server/src/main/java/org/apache/ambari/server/utils/StageUtils.java > b6287e6 > > ambari-server/src/test/java/org/apache/ambari/server/configuration/ConfigurationTest.java > e47fcd2 > > ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelperTest.java > 6bece66 > > ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerImplTest.java > e795d28 > > ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerTest.java > a1f28f1 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClientConfigResourceProviderTest.java > c5994c5 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProviderTest.java > 51035ba > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/HostStackVersionResourceProviderTest.java > 670c187 > > ambari-server/src/test/java/org/apache/ambari/server/utils/StageUtilsTest.java > b90295d > > > Diff: https://reviews.apache.org/r/64887/diff/1/ > > > Testing > --- > > tests ran on ambari-server project. > > > Thanks, > > Yussuf Shaikh > >
Re: Review Request 64769: AMBARI-22406 : UI for verifying if repo exist for new OS family and user prompt for entering HDP and HDP-UTILs repo URLs
> On Dec. 22, 2017, 9:51 a.m., Nate Cole wrote: > > ambari-web/app/controllers/wizard/step3_controller.js > > Lines 863-880 (patched) > > <https://reviews.apache.org/r/64769/diff/2/?file=1925966#file1925966line863> > > > > I'm no UI expert, but this entire new block seems to be doing repo > > stuff that already is on the UI. Can you explain the gist here? > > Sonia Garudi wrote: > We need each host's os type which is not available on the UI. This > function makes an API call which gets information of all the bootstrapped > hosts . I see. We should probably have that returned as part of host registration and read it from there instead of making another call. Is that possible in this patch? > On Dec. 22, 2017, 9:51 a.m., Nate Cole wrote: > > ambari-web/app/controllers/wizard/step3_controller.js > > Lines 968-972 (patched) > > <https://reviews.apache.org/r/64769/diff/2/?file=1925966#file1925966line968> > > > > Should not rely on the name for the id. The API should be giving you > > the id. > > Sonia Garudi wrote: > Made the required change in next diff. Ok, please resolve review issues as they are fixed in diffs with "Fixed" > On Dec. 22, 2017, 9:51 a.m., Nate Cole wrote: > > ambari-web/app/controllers/wizard/step3_controller.js > > Lines 1192 (patched) > > <https://reviews.apache.org/r/64769/diff/2/?file=1925966#file1925966line1192> > > > > ftp? I think it's fine to be only http or https. > > Sonia Garudi wrote: > Made the required change in next diff. Ok, please resolve review issues as they are fixed in diffs with "Fixed" - Nate --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64769/#review194434 --- On Dec. 26, 2017, 7:05 a.m., Sonia Garudi wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64769/ > --- > > (Updated Dec. 26, 2017, 7:05 a.m.) > > > Review request for Ambari and Nate Cole. > > > Bugs: AMBARI-22406 > https://issues.apache.org/jira/browse/AMBARI-22406 > > > Repository: ambari > > > Description > --- > > Includes UI changes in ambari-web to check if the repo is selected or > provides prompt to the user to enter the HDP and HDP-utils repo URL's. > Includes necessary validation checks for the repo urls, allows user to select > the public and local repository and also provides functionality to skip > validation checks. > > > Diffs > - > > ambari-web/app/controllers/wizard/step3_controller.js 29393e3 > ambari-web/app/styles/wizard.less 7d425d9 > ambari-web/app/templates/wizard/step3.hbs e00012c > ambari-web/app/utils/ajax/ajax.js 3f85dfdc > ambari-web/app/views/wizard/step3_view.js 3249b24 > ambari-web/test/controllers/wizard/step3_test.js b8bc794 > > > Diff: https://reviews.apache.org/r/64769/diff/3/ > > > Testing > --- > > Tests were successful in ambari-web > > > Thanks, > > Sonia Garudi > >
Re: Review Request 64815: For mixed OS deploy, some pig tests failed with MR job failing with 'IOException: Unable to get CompressorType for codec (org.apache.hadoop.io.compress.SnappyCodec)'
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64815/ --- (Updated Dec. 22, 2017, 4:43 p.m.) Review request for Ambari and Jonathan Hurley. Bugs: AMBARI-22694 https://issues.apache.org/jira/browse/AMBARI-22694 Repository: ambari Description --- Switch the ordering of the patch properties to do disk-first on the path, then the downloaded tarball Diffs (updated) - ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/configuration-mapred/mapred-site.xml 398c9d7d20 ambari-server/src/main/resources/stacks/HDP/2.2/services/TEZ/configuration/tez-site.xml 5513ab1a4a ambari-server/src/main/resources/stacks/HDP/2.2/services/YARN/configuration-mapred/mapred-site.xml 099e38866e ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/config-upgrade.xml 5b27fafe54 Diff: https://reviews.apache.org/r/64815/diff/2/ Changes: https://reviews.apache.org/r/64815/diff/1-2/ Testing --- No automated tests. Thanks, Nate Cole
Review Request 64815: For mixed OS deploy, some pig tests failed with MR job failing with 'IOException: Unable to get CompressorType for codec (org.apache.hadoop.io.compress.SnappyCodec)'
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64815/ --- Review request for Ambari and Jonathan Hurley. Bugs: AMBARI-22694 https://issues.apache.org/jira/browse/AMBARI-22694 Repository: ambari Description --- Switch the ordering of the patch properties to do disk-first on the path, then the downloaded tarball Diffs - ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/config-upgrade.xml 5b27fafe54 Diff: https://reviews.apache.org/r/64815/diff/1/ Testing --- No automated tests. Thanks, Nate Cole
Re: Review Request 64571: AMBARI-22404: Set java.home. in ambari-server setup script for server OS family
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64571/#review194439 --- Ship it! Ship It! - Nate Cole On Dec. 15, 2017, 7:23 a.m., Pravin Dsilva wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64571/ > --- > > (Updated Dec. 15, 2017, 7:23 a.m.) > > > Review request for Ambari and Nate Cole. > > > Bugs: AMBARI-22404 > https://issues.apache.org/jira/browse/AMBARI-22404 > > > Repository: ambari > > > Description > --- > > Changes to ambari-server setup script for accepting user values for Java Home > for OS type. > > > Diffs > - > > ambari-server/src/main/python/ambari-server.py 655a5d2 > ambari-server/src/main/python/ambari_server/serverSetup.py 13d5348 > ambari-server/src/test/python/TestAmbariServer.py 8ce2457 > > > Diff: https://reviews.apache.org/r/64571/diff/2/ > > > Testing > --- > > All tests passed > > > Thanks, > > Pravin Dsilva > >
Re: Review Request 64667: Livy server fails to start during downgrade due to absence of 'conf' directory
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64667/#review194438 --- What's the status of this? I thought we fixed all the outstanding Livy issues? - Nate Cole On Dec. 16, 2017, 10:53 a.m., Dmitro Lisnichenko wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64667/ > --- > > (Updated Dec. 16, 2017, 10:53 a.m.) > > > Review request for Ambari, Jonathan Hurley, Mahadev Konar, and Nate Cole. > > > Bugs: AMBARI-22522 > https://issues.apache.org/jira/browse/AMBARI-22522 > > > Repository: ambari > > > Description > --- > > Adding a small, livy-only workaround into install_packages.py > > > Diffs > - > > ambari-server/src/main/resources/custom_actions/scripts/install_packages.py > c8497cd > > > Diff: https://reviews.apache.org/r/64667/diff/1/ > > > Testing > --- > > mvn clean test, > live cluster check > > > Thanks, > > Dmitro Lisnichenko > >
Re: Review Request 64775: AMBARI-22394: Store and retrieve the ambari repo urls for os family during bootstrap
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64775/#review194436 --- ambari-server/src/main/python/bootstrap.py Lines 762-763 (patched) <https://reviews.apache.org/r/64775/#comment273203> Use "".format() syntax here. Should we give more information here? How will we tell the user what to do? How do they get the os_type into ambari.properties? - Nate Cole On Dec. 21, 2017, 10:25 a.m., Sneha Kanekar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64775/ > --- > > (Updated Dec. 21, 2017, 10:25 a.m.) > > > Review request for Ambari and Nate Cole. > > > Bugs: AMBARI-22394 > https://issues.apache.org/jira/browse/AMBARI-22394 > > > Repository: ambari > > > Description > --- > > Ambari configuration will have list of repo urls for each os type. During > bootstrap, the stored repo url will be used to correct the base url from > ambari repo file on agent host. > > > Diffs > - > > ambari-server/src/main/python/bootstrap.py f1c53ce > ambari-server/src/main/python/os_check_type.py da37560 > ambari-server/src/main/python/setupAgent.py efc3d7c > ambari-server/src/test/python/TestBootstrap.py bea47f4 > ambari-server/src/test/python/TestSetupAgent.py 0b33e9f > > > Diff: https://reviews.apache.org/r/64775/diff/1/ > > > Testing > --- > > Tests were successful in ambari-server. > > > Thanks, > > Sneha Kanekar > >
Re: Review Request 64769: AMBARI-22406 : UI for verifying if repo exist for new OS family and user prompt for entering HDP and HDP-UTILs repo URLs
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64769/#review194434 --- ambari-web/app/controllers/wizard/step3_controller.js Lines 863-880 (patched) <https://reviews.apache.org/r/64769/#comment273200> I'm no UI expert, but this entire new block seems to be doing repo stuff that already is on the UI. Can you explain the gist here? ambari-web/app/controllers/wizard/step3_controller.js Lines 968-972 (patched) <https://reviews.apache.org/r/64769/#comment273198> Should not rely on the name for the id. The API should be giving you the id. ambari-web/app/controllers/wizard/step3_controller.js Lines 1192 (patched) <https://reviews.apache.org/r/64769/#comment273199> ftp? I think it's fine to be only http or https. - Nate Cole On Dec. 21, 2017, 11:41 p.m., Sonia Garudi wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64769/ > --- > > (Updated Dec. 21, 2017, 11:41 p.m.) > > > Review request for Ambari and Nate Cole. > > > Bugs: AMBARI-22406 > https://issues.apache.org/jira/browse/AMBARI-22406 > > > Repository: ambari > > > Description > --- > > Includes UI changes in ambari-web to check if the repo is selected or > provides prompt to the user to enter the HDP and HDP-utils repo URL's. > Includes necessary validation checks for the repo urls, allows user to select > the public and local repository and also provides functionality to skip > validation checks. > > > Diffs > - > > ambari-web/app/controllers/wizard/step3_controller.js 29393e3 > ambari-web/app/styles/wizard.less 7d425d9 > ambari-web/app/templates/wizard/step3.hbs e00012c > ambari-web/app/utils/ajax/ajax.js 3f85dfdc > ambari-web/app/views/wizard/step3_view.js 3249b24 > ambari-web/test/controllers/wizard/step3_test.js b8bc794 > > > Diff: https://reviews.apache.org/r/64769/diff/2/ > > > Testing > --- > > Tests were successful in ambari-web > > > Thanks, > > Sonia Garudi > >
Re: Review Request 64785: Disabled stack still appears in the UI if VDF is available
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64785/#review194365 --- Ship it! Ship It! - Nate Cole On Dec. 21, 2017, 4 p.m., Sid Wagle wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64785/ > --- > > (Updated Dec. 21, 2017, 4 p.m.) > > > Review request for Ambari, Aravindan Vijayan, Jonathan Hurley, and Nate Cole. > > > Bugs: AMBARI-22686 > https://issues.apache.org/jira/browse/AMBARI-22686 > > > Repository: ambari > > > Description > --- > > Ambari-2.6.1, Ambari UI still shows HDP-2.3 as a stack that is active > although the stack definition marks this as disabled. > > Seems like the UI does not make any calls to check stack active/in-active. > The stacks returned from VDF endpoint are then used to get the repo URLs > because the VDF returns the stack def as sub-resource which included the > in-active stack. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java > 4ac8574 > > ambari-server/src/test/java/org/apache/ambari/server/api/services/AmbariMetaInfoTest.java > e9567f1 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/RepositoryVersionResourceProviderTest.java > 6bc8b95 > > > Diff: https://reviews.apache.org/r/64785/diff/1/ > > > Testing > --- > > Added a unit test. > Verified manually. > > > Thanks, > > Sid Wagle > >
Re: Review Request 64772: ServiceAdvisor for OneFS
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64772/#review194357 --- Ship it! Ship It! - Nate Cole On Dec. 21, 2017, 9:46 a.m., Attila Magyar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64772/ > --- > > (Updated Dec. 21, 2017, 9:46 a.m.) > > > Review request for Ambari, Attila Doroszlai, Jonathan Hurley, and Nate Cole. > > > Bugs: AMBARI-22674 > https://issues.apache.org/jira/browse/AMBARI-22674 > > > Repository: ambari > > > Description > --- > > A service advisor was added to the mpack to keep the hostnames of > core-site/fs.defaultFS, hdfs-site/dfs.namenode.http-address and > hdfs-site/dfs.namenode.https-address in sync. > > The user needs to enter the onefs/onefs_host and the service advisor will > update the other properties accordingly. > > > Diffs > - > > ambari-agent/src/main/python/ambari_agent/alerts/ams_alert.py 70f100f > ambari-agent/src/main/python/ambari_agent/alerts/base_alert.py 5c0305e > ambari-agent/src/main/python/ambari_agent/alerts/metric_alert.py 803bdc6 > ambari-agent/src/main/python/ambari_agent/alerts/port_alert.py 02cc91c > ambari-agent/src/main/python/ambari_agent/alerts/web_alert.py 0e400f7 > ambari-common/src/main/python/ambari_commons/inet_utils.py 5fd3e88 > contrib/management-packs/isilon-onefs-mpack/pom.xml 5d8f215 > > contrib/management-packs/isilon-onefs-mpack/src/main/resources/addon-services/ONEFS/1.0.0/configuration/core-site.xml > 7d3acd7 > > contrib/management-packs/isilon-onefs-mpack/src/main/resources/addon-services/ONEFS/1.0.0/configuration/hdfs-site.xml > cb6544f > > contrib/management-packs/isilon-onefs-mpack/src/main/resources/addon-services/ONEFS/1.0.0/configuration/onefs.xml > PRE-CREATION > > contrib/management-packs/isilon-onefs-mpack/src/main/resources/addon-services/ONEFS/1.0.0/metainfo.xml > 5ba562e > > contrib/management-packs/isilon-onefs-mpack/src/main/resources/addon-services/ONEFS/1.0.0/package/scripts/params_linux.py > 953efdc > > contrib/management-packs/isilon-onefs-mpack/src/main/resources/addon-services/ONEFS/1.0.0/service_advisor.py > PRE-CREATION > > contrib/management-packs/isilon-onefs-mpack/src/main/resources/addon-services/ONEFS/1.0.0/themes/theme.json > PRE-CREATION > > contrib/management-packs/isilon-onefs-mpack/src/test/python/TestServiceAdvisor.py > PRE-CREATION > contrib/management-packs/isilon-onefs-mpack/src/test/python/unitTests.py > PRE-CREATION > > > Diff: https://reviews.apache.org/r/64772/diff/1/ > > > Testing > --- > > - cluster creation, changed configs manually > > existing tests: passed > > > Thanks, > > Attila Magyar > >
Re: Review Request 63955: Validate blueprint does not allow lzo enable without setup with license agreement.
> On Dec. 13, 2017, 10:37 a.m., Nate Cole wrote: > > Has this been pushed? If so, please close the review. Ping - Nate --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63955/#review193685 --- On Nov. 20, 2017, 12:06 p.m., Myroslav Papirkovskyy wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63955/ > --- > > (Updated Nov. 20, 2017, 12:06 p.m.) > > > Review request for Ambari, Andrew Onischuk, Jonathan Hurley, Nate Cole, > Robert Nettleton, and Sid Wagle. > > > Bugs: AMBARI-22480 > https://issues.apache.org/jira/browse/AMBARI-22480 > > > Repository: ambari > > > Description > --- > > Blueprint validation should fail if LZO is enabled and GPL license is not > accepted on server setup. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintResourceProvider.java > 1c127c080cd33314e47bcfff63e0c1e6d58937bc > > ambari-server/src/main/java/org/apache/ambari/server/topology/Blueprint.java > c9dc7281a092ff9f22551c9f8fc64ebca15fda36 > > ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintImpl.java > 1d917e0940b1ba5988a740005017c9d228e46f96 > > ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintValidator.java > 206d161fc694f67ae982ef668c5057b4bacec59b > > ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintValidatorImpl.java > 1aaa5ce66d1a5385018e4f29c92441e157c2be02 > > ambari-server/src/main/java/org/apache/ambari/server/topology/GPLLicenseNotAcceptedException.java > PRE-CREATION > > ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintImplTest.java > fdbc0807788377f689e19200be9d6cdc2c992690 > > ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintValidatorImplTest.java > ea6bee314192f0665d4df652cf0c401e0218026f > > > Diff: https://reviews.apache.org/r/63955/diff/1/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Myroslav Papirkovskyy > >
Re: Review Request 64751: Fix Broken Symlinks on Stack Distribution
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64751/#review194353
---
Ship it!
Ship It!
- Nate Cole
On Dec. 21, 2017, 8:25 a.m., Dmitro Lisnichenko wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64751/
> ---
>
> (Updated Dec. 21, 2017, 8:25 a.m.)
>
>
> Review request for Ambari, Jonathan Hurley and Nate Cole.
>
>
> Bugs: AMBARI-22678
> https://issues.apache.org/jira/browse/AMBARI-22678
>
>
> Repository: ambari
>
>
> Description
> ---
>
> There are two scenarios to cover here:
>
> # Ambari never conf-select'd a component (maybe because of a bug or because
> the component didn't support it)
> # The conf pointers of a component are broken
>
> In either event, when distributing a new stack, the code detects this problem
> (as it would on a first-time install) and tries to fix it:
> {code}
> /etc/component/conf (directory)
> /usr/hdp/current/component -> /usr/hdp/v1/component
> /usr/hdp/v1/component -> /etc/component/conf
> {code}
>
> The stack distribution thinks this is a first-time installed and tries to fix
> the symlinks. We end up with:
> {code}
> /etc/component/conf -> /usr/hdp/current/component
> /usr/hdp/current/component -> /usr/hdp/v1/component
> /usr/hdp/v1/component -> /etc/component/conf
> /usr/hdp/v2/component -> /etc/component/v2/0
> {code}
>
> Because we're only conf-selecting v2, v1 never gets corrected since it's
> already installed. Thus, we have a circular symlink.
>
> Most likely the proper fix will be:
> - Iterate over the entire known conf-select structure
> - Check to see the state /etc/component/conf - if it's bad, fix it to defaults
>
> Chances are we can do this directly in
> {{conf_select.convert_conf_directories_to_symlinks}}:
> {code}
> stack_name = Script.get_stack_name()
> for directory_struct in dirs:
> if not os.path.exists(directory_struct['conf_dir']):
> Logger.info("Skipping the conf-select tool on {0} since {1} does not
> exist.".format(
> package, directory_struct['conf_dir']))
>
> return
> {code}
>
>
> Diffs
> -
>
>
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/livy_service.py
> cce2148ab6
>
> ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/livy2_service.py
> dfadd84613
> ambari-server/src/main/resources/custom_actions/scripts/install_packages.py
> c8497cd2cb
>
> ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_packages.json
> 62a46b91bd
>
>
> Diff: https://reviews.apache.org/r/64751/diff/2/
>
>
> Testing
> ---
>
> mvn clean test
>
> Live cluster check
>
>
> Thanks,
>
> Dmitro Lisnichenko
>
>
Review Request 64722: [Patch Hive]webhcat: test_sqoop fails with hdfs:///hdp/apps/2.6.*/sqoop/sqoop.tar.gz does not exist
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64722/ --- Review request for Ambari and Jonathan Hurley. Bugs: AMBARI-22676 https://issues.apache.org/jira/browse/AMBARI-22676 Repository: ambari Description --- Add SQOOP to the list of services that should go with HIVE Diffs - ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_packages.json 62a46b91bd Diff: https://reviews.apache.org/r/64722/diff/1/ Testing --- No new tests for json change. Thanks, Nate Cole
Re: Review Request 64544: Migrate user data for upgrade to improved user account management
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64544/#review194085 --- Ship it! Ship It! - Nate Cole On Dec. 13, 2017, 5:01 p.m., Robert Levas wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64544/ > --- > > (Updated Dec. 13, 2017, 5:01 p.m.) > > > Review request for Ambari, Attila Magyar, Balázs Bence Sári, Eugene > Chekanskiy, Jonathan Hurley, Nate Cole, Robert Nettleton, and Sandor Molnar. > > > Bugs: AMBARI-22577 > https://issues.apache.org/jira/browse/AMBARI-22577 > > > Repository: ambari > > > Description > --- > > Migrate data from the users table (pre-Ambari 3.0.0) to the updated users > table and user_authentication tables. > > > Diffs > - > > ambari-server/src/main/java/org/apache/ambari/server/orm/DBAccessor.java > 549c0fd7e8 > > ambari-server/src/main/java/org/apache/ambari/server/orm/dao/UserAuthenticationDAO.java > 513e78200d > > ambari-server/src/main/java/org/apache/ambari/server/orm/entities/UserAuthenticationEntity.java > 27514f648c > > ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AmbariLdapAuthenticationProvider.java > 20a06ccd54 > > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog300.java > 2de60957bc > ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql e58a04e4b2 > ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql cc589e47f2 > ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 7cd083db4c > ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql ff232ebe3e > ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql > d7c09f3381 > ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql b4929b308d > > ambari-server/src/test/java/org/apache/ambari/server/orm/DBAccessorImplTest.java > 29f9d917e8 > > ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariLdapAuthenticationProviderForDNWithSpaceTest.java > 10076b0876 > > ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariLdapAuthenticationProviderTest.java > 43d4d6b0c0 > > ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog300Test.java > 747f99b618 > > > Diff: https://reviews.apache.org/r/64544/diff/2/ > > > Testing > --- > > manually tested > > # Local test results: > ``` > [INFO] > > [INFO] BUILD SUCCESS > [INFO] > > [INFO] Total time: 27:21 min > [INFO] Finished at: 2017-12-11T17:30:31-05:00 > [INFO] Final Memory: 100M/2019M > [INFO] > > ``` > > # Jenkins test results: PENDING > > > Thanks, > > Robert Levas > >
Re: Review Request 64652: unable to proceed with cluster install after component install fails.
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64652/#review193962
---
Ship it!
Ship It!
- Nate Cole
On Dec. 15, 2017, noon, Myroslav Papirkovskyy wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64652/
> ---
>
> (Updated Dec. 15, 2017, noon)
>
>
> Review request for Ambari, Dmitro Lisnichenko, Jonathan Hurley, and Nate Cole.
>
>
> Bugs: AMBARI-22659
> https://issues.apache.org/jira/browse/AMBARI-22659
>
>
> Repository: ambari
>
>
> Description
> ---
>
> Unable go back in install wizard if install was finished at one of hosts.
>
> ```
> Request
> URL:http://ambari:8080/api/v1/stacks/HDP/versions/2.6/repository_versions/1
>
> {
> "status" : 500,
> "message" : "org.apache.ambari.server.controller.spi.SystemException:
> Repository version can't be deleted as it is used by the following hosts:
> CURRENT on sanjay-divgi-test-2.openstacklocal,
> sanjay-divgi-test-5.openstacklocal, sanjay-divgi-test-1.openstacklocal,
> sanjay-divgi-test-4.openstacklocal"
> }
> ```
>
>
> Diffs
> -
>
>
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostVersionDAO.java
> 158370f3528f796e30a46862e8eb42882001ca02
>
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/HostVersionEntity.java
> 4a030af122dbe3ceb3c3592c000b126c864dc626
>
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
> c23b971b85a843433f230b5a94c66ac46c5b1914
>
> ambari-server/src/test/java/org/apache/ambari/server/orm/dao/HostVersionDAOTest.java
> bb077d6667138acdaeaafa8b07f188b2ce3dc2af
>
>
> Diff: https://reviews.apache.org/r/64652/diff/1/
>
>
> Testing
> ---
>
> mvn clean test
> manual check
>
>
> Thanks,
>
> Myroslav Papirkovskyy
>
>
Re: Review Request 64652: unable to proceed with cluster install after component install fails.
> On Dec. 15, 2017, 2:01 p.m., Nate Cole wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
> > Lines 1357-1364 (patched)
> > <https://reviews.apache.org/r/64652/diff/1/?file=1918447#file1918447line1357>
> >
> > These records should just be straight up deleted, not marked this way.
>
> Myroslav Papirkovskyy wrote:
> UI makes explicit call to delete host versions, which will fail if we
> just remove records.
>
> Jonathan Hurley wrote:
> If the UI is deleting the records, why do we even need this?
>
> Myroslav Papirkovskyy wrote:
> Because we cannot remove CURRENT version. This state is set if at least
> one host finished install for all components.
>
> Jonathan Hurley wrote:
> Gotcha...
Got it. Looks good to me! Ship it.
- Nate
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64652/#review193951
---
On Dec. 15, 2017, noon, Myroslav Papirkovskyy wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64652/
> ---
>
> (Updated Dec. 15, 2017, noon)
>
>
> Review request for Ambari, Dmitro Lisnichenko, Jonathan Hurley, and Nate Cole.
>
>
> Bugs: AMBARI-22659
> https://issues.apache.org/jira/browse/AMBARI-22659
>
>
> Repository: ambari
>
>
> Description
> ---
>
> Unable go back in install wizard if install was finished at one of hosts.
>
> ```
> Request
> URL:http://ambari:8080/api/v1/stacks/HDP/versions/2.6/repository_versions/1
>
> {
> "status" : 500,
> "message" : "org.apache.ambari.server.controller.spi.SystemException:
> Repository version can't be deleted as it is used by the following hosts:
> CURRENT on sanjay-divgi-test-2.openstacklocal,
> sanjay-divgi-test-5.openstacklocal, sanjay-divgi-test-1.openstacklocal,
> sanjay-divgi-test-4.openstacklocal"
> }
> ```
>
>
> Diffs
> -
>
>
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostVersionDAO.java
> 158370f3528f796e30a46862e8eb42882001ca02
>
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/HostVersionEntity.java
> 4a030af122dbe3ceb3c3592c000b126c864dc626
>
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
> c23b971b85a843433f230b5a94c66ac46c5b1914
>
> ambari-server/src/test/java/org/apache/ambari/server/orm/dao/HostVersionDAOTest.java
> bb077d6667138acdaeaafa8b07f188b2ce3dc2af
>
>
> Diff: https://reviews.apache.org/r/64652/diff/1/
>
>
> Testing
> ---
>
> mvn clean test
> manual check
>
>
> Thanks,
>
> Myroslav Papirkovskyy
>
>
Re: Review Request 64652: unable to proceed with cluster install after component install fails.
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64652/#review193951
---
ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
Lines 1357-1364 (patched)
<https://reviews.apache.org/r/64652/#comment272627>
These records should just be straight up deleted, not marked this way.
- Nate Cole
On Dec. 15, 2017, noon, Myroslav Papirkovskyy wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64652/
> ---
>
> (Updated Dec. 15, 2017, noon)
>
>
> Review request for Ambari, Dmitro Lisnichenko, Jonathan Hurley, and Nate Cole.
>
>
> Bugs: AMBARI-22659
> https://issues.apache.org/jira/browse/AMBARI-22659
>
>
> Repository: ambari
>
>
> Description
> ---
>
> Unable go back in install wizard if install was finished at one of hosts.
>
> ```
> Request
> URL:http://ambari:8080/api/v1/stacks/HDP/versions/2.6/repository_versions/1
>
> {
> "status" : 500,
> "message" : "org.apache.ambari.server.controller.spi.SystemException:
> Repository version can't be deleted as it is used by the following hosts:
> CURRENT on sanjay-divgi-test-2.openstacklocal,
> sanjay-divgi-test-5.openstacklocal, sanjay-divgi-test-1.openstacklocal,
> sanjay-divgi-test-4.openstacklocal"
> }
> ```
>
>
> Diffs
> -
>
>
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostVersionDAO.java
> 158370f3528f796e30a46862e8eb42882001ca02
>
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/HostVersionEntity.java
> 4a030af122dbe3ceb3c3592c000b126c864dc626
>
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
> c23b971b85a843433f230b5a94c66ac46c5b1914
>
> ambari-server/src/test/java/org/apache/ambari/server/orm/dao/HostVersionDAOTest.java
> bb077d6667138acdaeaafa8b07f188b2ce3dc2af
>
>
> Diff: https://reviews.apache.org/r/64652/diff/1/
>
>
> Testing
> ---
>
> mvn clean test
> manual check
>
>
> Thanks,
>
> Myroslav Papirkovskyy
>
>
Re: Review Request 64637: Livy/Livy2 Unable To Start Due to Address Already In Use
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64637/#review193918
---
Ship it!
Ship It!
- Nate Cole
On Dec. 14, 2017, 10:38 p.m., Jonathan Hurley wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64637/
> ---
>
> (Updated Dec. 14, 2017, 10:38 p.m.)
>
>
> Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, and Nate Cole.
>
>
> Bugs: AMBARI-22655
> https://issues.apache.org/jira/browse/AMBARI-22655
>
>
> Repository: ambari
>
>
> Description
> ---
>
> While restarting Livy and Livy2 on a non-root cluster, the following is seen:
>
> ```
> 17/12/14 14:36:23 WARN LivyConf: The configuration key
> livy.repl.enableHiveContext has been deprecated as of Livy 0.4 and may be
> removed in the future. Please use the new key livy.repl.enable-hive-context
> instead.
> 17/12/14 14:36:23 WARN LivyConf: The configuration key
> livy.server.csrf_protection.enabled has been deprecated as of Livy 0.4 and
> may be removed in the future. Please use the new key
> livy.server.csrf-protection.enabled instead.
> 17/12/14 14:36:23 INFO AccessManager: AccessControlManager acls
> disabled;users with view permission: ;users with modify permission: ;users
> with super permission: cstm-zeppelin;other allowed users: *
> 17/12/14 14:36:28 INFO LineBufferedStream: stdout: Welcome to
> 17/12/14 14:36:28 INFO LineBufferedStream: stdout: __
> 17/12/14 14:36:28 INFO LineBufferedStream: stdout: / __/__ ___ _/
> /__
> 17/12/14 14:36:28 INFO LineBufferedStream: stdout: _\ \/ _ \/ _ `/ __/
> '_/
> 17/12/14 14:36:28 INFO LineBufferedStream: stdout:/___/ .__/_,_/_/ /_/_\
> version 2.2.0.2.6.4.0-73
> 17/12/14 14:36:28 INFO LineBufferedStream: stdout: /_/
> 17/12/14 14:36:28 INFO LineBufferedStream: stdout:
> 17/12/14 14:36:28 INFO LineBufferedStream: stdout: Using Scala version
> 2.11.8, OpenJDK 64-Bit Server VM, 1.8.0_131
> 17/12/14 14:36:28 INFO LineBufferedStream: stdout: Branch HEAD
> 17/12/14 14:36:28 INFO LineBufferedStream: stdout: Compiled by user jenkins
> on 2017-12-13T19:08:32Z
> 17/12/14 14:36:28 INFO LineBufferedStream: stdout: Revision
> a24017869f5450397136ee8b11be818e7cd3facb
> 17/12/14 14:36:28 INFO LineBufferedStream: stdout: Url
> g...@github.com:hortonworks/spark2.git
> 17/12/14 14:36:28 INFO LineBufferedStream: stdout: Type --help for more
> information.
> 17/12/14 14:36:29 WARN NativeCodeLoader: Unable to load native-hadoop library
> for your platform... using builtin-java classes where applicable
> 17/12/14 14:36:31 INFO AHSProxy: Connecting to Application History server at
> nat-yc-r7-ovvs-ambari-autostart-4-re-2.openstacklocal/172.22.121.144:10200
> 17/12/14 14:36:32 WARN DomainSocketFactory: The short-circuit local reads
> feature cannot be used because libhadoop cannot be loaded.
> 17/12/14 14:36:33 INFO StateStore$: Using FileSystemStateStore for recovery.
> 17/12/14 14:36:33 INFO BatchSessionManager: Recovered 0 batch sessions. Next
> session id: 0
> 17/12/14 14:36:33 INFO InteractiveSessionManager: Recovered 0 interactive
> sessions. Next session id: 0
> 17/12/14 14:36:33 INFO InteractiveSessionManager: Heartbeat watchdog thread
> started.
> 17/12/14 14:36:33 INFO LivyServer: SPNEGO auth enabled (principal =
> HTTP/nat-yc-r7-ovvs-ambari-autostart-4-re-3.openstacklo...@example.com)
> 17/12/14 14:36:33 INFO LivyServer: CSRF protection is enabled.
> 17/12/14 14:36:34 INFO KerberosAuthenticationHandler: Login using keytab
> /etc/security/keytabs/spnego.service.keytab, for principal
> HTTP/nat-yc-r7-ovvs-ambari-autostart-4-re-3.openstacklo...@example.com
> 17/12/14 14:36:34 INFO KerberosAuthenticationHandler: Map server:
> nat-yc-r7-ovvs-ambari-autostart-4-re-3.openstacklocal to principal:
> [HTTP/nat-yc-r7-ovvs-ambari-autostart-4-re-3.openstacklo...@example.com],
> added = true
> 17/12/14 14:36:34 WARN AbstractLifeCycle: FAILED
> ServerConnector@df1cff6{SSL-http/1.1}{0.0.0.0:8999}: java.net.BindException:
> Address already in use
> java.net.BindException: Address already in use
> at sun.nio.ch.Net.bind0(Native Method)
> ```
>
> This occurs because the PID file cannot be accessed by the non-root agent and
> returns an exit code of 1:
> {code}
> call returned (1, 'cat: /var/run/livy/livy-cstm-livy-server.pid: Permission
> denied')
> {code}
>
> This tricks out PID detection into thinking that
Re: Review Request 64542: Kerberos support for OneFS
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64542/#review193791 --- Ship it! Ship It! - Nate Cole On Dec. 14, 2017, 6:26 a.m., Attila Magyar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64542/ > --- > > (Updated Dec. 14, 2017, 6:26 a.m.) > > > Review request for Ambari, Andrii Babiichuk, Eugene Chekanskiy, Nate Cole, > Robert Levas, and Sandor Molnar. > > > Bugs: AMBARI-22634 > https://issues.apache.org/jira/browse/AMBARI-22634 > > > Repository: ambari > > > Description > --- > > When kerberos is enabled, the OneFS service check should do a kinit before > executing hdfs commands. I also removed some unused code from the > servicecheck script. > > > Diffs > - > > ambari-web/app/controllers/main/admin/kerberos/step1_controller.js 9c864a8 > ambari-web/app/messages.js b305196 > ambari-web/app/templates/main/admin/kerberos/step1.hbs 57d0637 > > contrib/management-packs/isilon-onefs-mpack/src/main/resources/addon-services/ONEFS/1.0.0/kerberos.json > PRE-CREATION > > contrib/management-packs/isilon-onefs-mpack/src/main/resources/addon-services/ONEFS/1.0.0/package/scripts/params_linux.py > b823fc5 > > contrib/management-packs/isilon-onefs-mpack/src/main/resources/addon-services/ONEFS/1.0.0/package/scripts/service_check.py > 3d798a3 > > > Diff: https://reviews.apache.org/r/64542/diff/3/ > > > Testing > --- > > - enabled kerberos, ran service checks on onefs > > > Thanks, > > Attila Magyar > >
Re: Review Request 64579: Node Managers fail to start after Spark2 is patched due to CNF YarnShuffleService
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64579/#review193703
---
Ship it!
Ship It!
- Nate Cole
On Dec. 13, 2017, 12:12 p.m., Jonathan Hurley wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64579/
> ---
>
> (Updated Dec. 13, 2017, 12:12 p.m.)
>
>
> Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, and Nate Cole.
>
>
> Bugs: AMBARI-22644
> https://issues.apache.org/jira/browse/AMBARI-22644
>
>
> Repository: ambari
>
>
> Description
> ---
>
> *STR*
> # Deploy HDP-2.6.4.0 cluster with Ambari-2.6.1.0-114
> # Apply HBase patch Upgrade on the cluster (this step is optional)
> # Then apply Spark2 patch Upgrade on the cluster
> # Restart Node Managers
>
> *Result*
> NM restart fails with below error:
> ```
> 2017-12-10 07:17:02,559 INFO impl.MetricsSystemImpl
> (MetricsSystemImpl.java:shutdown(606)) - NodeManager metrics system shutdown
> complete.
> 2017-12-10 07:17:02,559 FATAL nodemanager.NodeManager
> (NodeManager.java:initAndStartNodeManager(549)) - Error starting NodeManager
> Caused by: java.lang.ClassNotFoundException:
> org.apache.spark.network.yarn.YarnShuffleService
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> at
> org.apache.hadoop.util.ApplicationClassLoader.loadClass(ApplicationClassLoader.java:197)
> at
> org.apache.hadoop.util.ApplicationClassLoader.loadClass(ApplicationClassLoader.java:165)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:348)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxiliaryServiceWithCustomClassLoader.getInstance(AuxiliaryServiceWithCustomClassLoader.java:169)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices.serviceInit(AuxServices.java:131)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> ... 8 more
> 2017-12-10 07:17:02,562 INFO nodemanager.NodeManager
> (LogAdapter.java:info(45)) - SHUTDOWN_MSG:
> ```
>
> The spark properties are correctly being written out as per AMBARI-22525.
>
> Initially, we had defined Spark properties for ATS like this:
> ```xml
> yarn.nodemanager.aux-services.spark_shuffle.classpath
> {{stack_root}}/${hdp.version}/spark/aux/*
> ```
>
> When YARN upgrades without Spark, we run into AMBARI-22525. Seems like the
> shuffle classes are installed as part of RPM dependencies, but not the
> SparkATSPlugin.
>
> So:
> - If we use YARN's version for the Spark classes, then ATS can't find
> SparkATSPlugin since that is not part of YARN.
> - If we use Spark's version for the classes, then Spark can never upgrade
> without YARN since NodeManager can't find the new Spark classes.
>
> However, it seems like shuffle and ATS use different properties. We changed
> all 3 properties in AMBARI-22525:
>
> ```
> yarn.nodemanager.aux-services.spark2_shuffle.classpath
> yarn.nodemanager.aux-services.spark_shuffle.classpath
> yarn.timeline-service.entity-group-fs-store.group-id-plugin-classpath
> ```
>
> It seems like what need to do is change the spark shuffle stuff back to
> hdp.version, but leave ATS using the new version since we're guaranteed to
> have Spark installed on the ATS machine.
>
>
> Diffs
> -
>
>
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py
> 6b5559cf91
>
> ambari-server/src/main/resources/stacks/HDP/2.5/services/YARN/configuration/yarn-site.xml
> 29833fbe03
> ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/config-upgrade.xml
> ea0e2cd46b
>
>
> Diff: https://reviews.apache.org/r/64579/diff/2/
>
>
> Testing
> ---
>
> Manual testing on a patched cluster with YARN/Spark
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 64554: HBase Cannot Find LZO Classes After Being Patched
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64554/#review193702 --- Ship it! Ship It! - Nate Cole On Dec. 13, 2017, 10:46 a.m., Jonathan Hurley wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64554/ > --- > > (Updated Dec. 13, 2017, 10:46 a.m.) > > > Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, and Nate Cole. > > > Bugs: AMBARI-22640 > https://issues.apache.org/jira/browse/AMBARI-22640 > > > Repository: ambari > > > Description > --- > > After patching HBase where LZO compression is being used, the following is > seen: > > ``` > 2017-12-10 22:31:09,244|INFO|MainThread|machine.py:164 - > run()||GUID=37b565a7-e164-4641-b335-19884c614ffd|Exception in thread "main" > java.lang.RuntimeException: java.lang.ClassNotFoundException: > com.hadoop.compression.lzo.LzoCodec > 2017-12-10 22:31:09,245|INFO|MainThread|machine.py:164 - > run()||GUID=37b565a7-e164-4641-b335-19884c614ffd|at > org.apache.hadoop.hbase.io.compress.Compression$Algorithm$1.buildCodec(Compression.java:130) > 2017-12-10 22:31:09,245|INFO|MainThread|machine.py:164 - > run()||GUID=37b565a7-e164-4641-b335-19884c614ffd|at > org.apache.hadoop.hbase.io.compress.Compression$Algorithm$1.getCodec(Compression.java:116) > 2017-12-10 22:31:09,245|INFO|MainThread|machine.py:164 - > run()||GUID=37b565a7-e164-4641-b335-19884c614ffd|at > org.apache.hadoop.hbase.io.compress.Compression$Algorithm.getCompressor(Compression.java:301) > 2017-12-10 22:31:09,245|INFO|MainThread|machine.py:164 - > run()||GUID=37b565a7-e164-4641-b335-19884c614ffd|at > org.apache.hadoop.hbase.io.encoding.HFileBlockDefaultEncodingContext.2017-12-10 > 22:31:09,267|INFO|MainThread|machine.py:164 - > run()||GUID=37b565a7-e164-4641-b335-19884c614ffd|at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > 2017-12-10 22:31:09,268|INFO|MainThread|machine.py:164 - > run()||GUID=37b565a7-e164-4641-b335-19884c614ffd|at > java.util.concurrent.FutureTask.run(FutureTask.java:266) > 2017-12-10 22:31:09,268|INFO|MainThread|machine.py:164 - > run()||GUID=37b565a7-e164-4641-b335-19884c614ffd|at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > 2017-12-10 22:31:09,268|INFO|MainThread|machine.py:164 - > run()||GUID=37b565a7-e164-4641-b335-19884c614ffd|at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > 2017-12-10 22:31:09,268|INFO|MainThread|machine.py:164 - > run()||GUID=37b565a7-e164-4641-b335-19884c614ffd|at > java.lang.Thread.run(Thread.java:745) > 2017-12-10 22:31:09,741|INFO|MainThread|machine.py:189 - > run()||GUID=37b565a7-e164-4641-b335-19884c614ffd|Exit Code: 1 > ``` > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/hbase.py > ac71ce4b36 > > > Diff: https://reviews.apache.org/r/64554/diff/2/ > > > Testing > --- > > Manual testing on Hbase cluster. > > > Thanks, > > Jonathan Hurley > >
Re: Review Request 63955: Validate blueprint does not allow lzo enable without setup with license agreement.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63955/#review193685 --- Has this been pushed? If so, please close the review. - Nate Cole On Nov. 20, 2017, 12:06 p.m., Myroslav Papirkovskyy wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63955/ > --- > > (Updated Nov. 20, 2017, 12:06 p.m.) > > > Review request for Ambari, Andrew Onischuk, Jonathan Hurley, Nate Cole, > Robert Nettleton, and Sid Wagle. > > > Bugs: AMBARI-22480 > https://issues.apache.org/jira/browse/AMBARI-22480 > > > Repository: ambari > > > Description > --- > > Blueprint validation should fail if LZO is enabled and GPL license is not > accepted on server setup. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintResourceProvider.java > 1c127c080cd33314e47bcfff63e0c1e6d58937bc > > ambari-server/src/main/java/org/apache/ambari/server/topology/Blueprint.java > c9dc7281a092ff9f22551c9f8fc64ebca15fda36 > > ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintImpl.java > 1d917e0940b1ba5988a740005017c9d228e46f96 > > ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintValidator.java > 206d161fc694f67ae982ef668c5057b4bacec59b > > ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintValidatorImpl.java > 1aaa5ce66d1a5385018e4f29c92441e157c2be02 > > ambari-server/src/main/java/org/apache/ambari/server/topology/GPLLicenseNotAcceptedException.java > PRE-CREATION > > ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintImplTest.java > fdbc0807788377f689e19200be9d6cdc2c992690 > > ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintValidatorImplTest.java > ea6bee314192f0665d4df652cf0c401e0218026f > > > Diff: https://reviews.apache.org/r/63955/diff/1/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Myroslav Papirkovskyy > >
Re: Review Request 63957: Stack advisor should disallow lzo enable without accepting license agreement.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63957/#review193684 --- Has this been pushed? If so, please close the review. - Nate Cole On Nov. 22, 2017, 1:09 p.m., Myroslav Papirkovskyy wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63957/ > --- > > (Updated Nov. 22, 2017, 1:09 p.m.) > > > Review request for Ambari, Andrew Onischuk, Jonathan Hurley, Nate Cole, and > Sid Wagle. > > > Bugs: AMBARI-22484 > https://issues.apache.org/jira/browse/AMBARI-22484 > > > Repository: ambari > > > Description > --- > > If the user adds com.hadoop.compression.lzo.LzoCodec to core-site property > io.compression.codec.lzo.class and the ambari.properties does not have the > allow GPL software install flag, throw validation error > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/StackAdvisorRequest.java > 64180e5039bdb74d3aaf25e140565c82a95a26c1 > > ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/commands/StackAdvisorCommand.java > 5440462b86cdcfafb1e9ab239dacfddee0b31b93 > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java > b2a68d66d777f107fdc4f5576a691e8d25ec2c92 > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackAdvisorResourceProvider.java > dd32c6badb18c1a8c02b7093813cd92ccd2fc74c > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py > c36a41f49b224a904368c7474f0f769348e65ae6 > ambari-server/src/main/resources/stacks/stack_advisor.py > 8ccbaba9db083068268951ae4e07ce08d32edcc4 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ValidationResourceProviderTest.java > cdbf2d782f86b2a30f54619c4e2a9165a020275c > ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py > 9e0070458987ee490f3918fead5ef11d201c8612 > > > Diff: https://reviews.apache.org/r/63957/diff/2/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Myroslav Papirkovskyy > >
Re: Review Request 64554: HBase Cannot Find LZO Classes After Being Patched
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64554/#review193680 --- ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/hbase.py Lines 29-31 (original), 41-47 (patched) <https://reviews.apache.org/r/64554/#comment272285> Only in WINSRV? - Nate Cole On Dec. 13, 2017, 9:02 a.m., Jonathan Hurley wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64554/ > --- > > (Updated Dec. 13, 2017, 9:02 a.m.) > > > Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, and Nate Cole. > > > Bugs: AMBARI-22640 > https://issues.apache.org/jira/browse/AMBARI-22640 > > > Repository: ambari > > > Description > --- > > After patching HBase where LZO compression is being used, the following is > seen: > > ``` > 2017-12-10 22:31:09,244|INFO|MainThread|machine.py:164 - > run()||GUID=37b565a7-e164-4641-b335-19884c614ffd|Exception in thread "main" > java.lang.RuntimeException: java.lang.ClassNotFoundException: > com.hadoop.compression.lzo.LzoCodec > 2017-12-10 22:31:09,245|INFO|MainThread|machine.py:164 - > run()||GUID=37b565a7-e164-4641-b335-19884c614ffd|at > org.apache.hadoop.hbase.io.compress.Compression$Algorithm$1.buildCodec(Compression.java:130) > 2017-12-10 22:31:09,245|INFO|MainThread|machine.py:164 - > run()||GUID=37b565a7-e164-4641-b335-19884c614ffd|at > org.apache.hadoop.hbase.io.compress.Compression$Algorithm$1.getCodec(Compression.java:116) > 2017-12-10 22:31:09,245|INFO|MainThread|machine.py:164 - > run()||GUID=37b565a7-e164-4641-b335-19884c614ffd|at > org.apache.hadoop.hbase.io.compress.Compression$Algorithm.getCompressor(Compression.java:301) > 2017-12-10 22:31:09,245|INFO|MainThread|machine.py:164 - > run()||GUID=37b565a7-e164-4641-b335-19884c614ffd|at > org.apache.hadoop.hbase.io.encoding.HFileBlockDefaultEncodingContext.2017-12-10 > 22:31:09,267|INFO|MainThread|machine.py:164 - > run()||GUID=37b565a7-e164-4641-b335-19884c614ffd|at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > 2017-12-10 22:31:09,268|INFO|MainThread|machine.py:164 - > run()||GUID=37b565a7-e164-4641-b335-19884c614ffd|at > java.util.concurrent.FutureTask.run(FutureTask.java:266) > 2017-12-10 22:31:09,268|INFO|MainThread|machine.py:164 - > run()||GUID=37b565a7-e164-4641-b335-19884c614ffd|at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > 2017-12-10 22:31:09,268|INFO|MainThread|machine.py:164 - > run()||GUID=37b565a7-e164-4641-b335-19884c614ffd|at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > 2017-12-10 22:31:09,268|INFO|MainThread|machine.py:164 - > run()||GUID=37b565a7-e164-4641-b335-19884c614ffd|at > java.lang.Thread.run(Thread.java:745) > 2017-12-10 22:31:09,741|INFO|MainThread|machine.py:189 - > run()||GUID=37b565a7-e164-4641-b335-19884c614ffd|Exit Code: 1 > ``` > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/hbase.py > ac71ce4b36 > > > Diff: https://reviews.apache.org/r/64554/diff/1/ > > > Testing > --- > > Manual testing on Hbase cluster. > > > Thanks, > > Jonathan Hurley > >
Re: Review Request 64571: AMBARI-22404: Set java.home. in ambari-server setup script for server OS family
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64571/#review193682 --- ambari-server/src/main/python/ambari_server/serverSetup.py Lines 870-876 (patched) <https://reviews.apache.org/r/64571/#comment272288> This appears to always force someone to pick an OS and changes the expectation of how the server gets installed. We shouldn't be changing how things are working today. You should do either: - After picking JAVA_HOME, then we should ask if they want to customize the OS type (as a choice). - Use another command line option to add a JAVA_HOME for an OS type. No need to ask the type, just use whatever value they put in the option. This way, you only have to check for the option to know to write "java.home.redhat7". No option means we maintain current behavior. I would prefer the 2nd option as it's likely easier to document. If we start asking "hard choices" the everyday user will get confused. - Nate Cole On Dec. 13, 2017, 4:28 a.m., Pravin Dsilva wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64571/ > --- > > (Updated Dec. 13, 2017, 4:28 a.m.) > > > Review request for Ambari and Nate Cole. > > > Bugs: AMBARI-22404 > https://issues.apache.org/jira/browse/AMBARI-22404 > > > Repository: ambari > > > Description > --- > > Changes to ambari-server setup script for accepting user values for Java Home > for OS type. > > > Diffs > - > > ambari-server/src/main/python/ambari_server/serverSetup.py 13d5348 > ambari-server/src/test/python/TestAmbariServer.py 8ce2457 > > > Diff: https://reviews.apache.org/r/64571/diff/1/ > > > Testing > --- > > All tests passed > > > Thanks, > > Pravin Dsilva > >
Re: Review Request 64544: Migrate user data for upgrade to improved user account management
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64544/#review193553 --- Ship it! Ship It! - Nate Cole On Dec. 12, 2017, 9:35 a.m., Robert Levas wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64544/ > --- > > (Updated Dec. 12, 2017, 9:35 a.m.) > > > Review request for Ambari, Attila Magyar, Balázs Bence Sári, Eugene > Chekanskiy, Jonathan Hurley, Nate Cole, Robert Nettleton, and Sandor Molnar. > > > Bugs: AMBARI-22577 > https://issues.apache.org/jira/browse/AMBARI-22577 > > > Repository: ambari > > > Description > --- > > Migrate data from the users table (pre-Ambari 3.0.0) to the updated users > table and user_authentication tables. > > > Diffs > - > > ambari-server/src/main/java/org/apache/ambari/server/orm/DBAccessor.java > 549c0fd7e8 > > ambari-server/src/main/java/org/apache/ambari/server/orm/dao/UserAuthenticationDAO.java > 513e78200d > > ambari-server/src/main/java/org/apache/ambari/server/orm/entities/UserAuthenticationEntity.java > 27514f648c > > ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AmbariLdapAuthenticationProvider.java > 20a06ccd54 > > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog300.java > 2de60957bc > ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql e58a04e4b2 > ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql cc589e47f2 > ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 7cd083db4c > ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql ff232ebe3e > ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql > d7c09f3381 > ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql b4929b308d > > ambari-server/src/test/java/org/apache/ambari/server/orm/DBAccessorImplTest.java > 29f9d917e8 > ambari-server/src/test/java/org/apache/ambari/server/orm/db/DDLTests.java > 96cf64e53c > > ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariLdapAuthenticationProviderForDNWithSpaceTest.java > 10076b0876 > > ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariLdapAuthenticationProviderTest.java > 43d4d6b0c0 > > ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog300Test.java > 747f99b618 > > > Diff: https://reviews.apache.org/r/64544/diff/1/ > > > Testing > --- > > manually tested > > # Local test results: > ``` > [INFO] > > [INFO] BUILD SUCCESS > [INFO] > > [INFO] Total time: 27:21 min > [INFO] Finished at: 2017-12-11T17:30:31-05:00 > [INFO] Final Memory: 100M/2019M > [INFO] > > ``` > > # Jenkins test results: PENDING > > > Thanks, > > Robert Levas > >
Re: Review Request 64542: Kerberos support for OneFS
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64542/#review193551 --- Fix it, then Ship it! ambari-web/app/controllers/main/admin/kerberos/step1_controller.js Lines 45-47 (patched) <https://reviews.apache.org/r/64542/#comment272119> Is this, or can this be made to show conditionally if isilon is chosen? - Nate Cole On Dec. 12, 2017, 10:38 a.m., Attila Magyar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64542/ > --- > > (Updated Dec. 12, 2017, 10:38 a.m.) > > > Review request for Ambari, Eugene Chekanskiy, Nate Cole, Robert Levas, and > Sandor Molnar. > > > Bugs: AMBARI-22634 > https://issues.apache.org/jira/browse/AMBARI-22634 > > > Repository: ambari > > > Description > --- > > When kerberos is enabled, the OneFS service check should do a kinit before > executing hdfs commands. I also removed some unused code from the > servicecheck script. > > > Diffs > - > > ambari-web/app/controllers/main/admin/kerberos/step1_controller.js > 9c864a8f48889515d5b72c0feaf340b7b9e1fe31 > ambari-web/app/messages.js b3051962f56951fc6bae169b63c5f029c1ef301d > > contrib/management-packs/isilon-onefs-mpack/src/main/resources/addon-services/ONEFS/1.0.0/kerberos.json > PRE-CREATION > > contrib/management-packs/isilon-onefs-mpack/src/main/resources/addon-services/ONEFS/1.0.0/package/scripts/params_linux.py > b823fc54b84e1149164e17504046a7a32070df1c > > contrib/management-packs/isilon-onefs-mpack/src/main/resources/addon-services/ONEFS/1.0.0/package/scripts/service_check.py > 3d798a3342ced3d99a4a2c955a6aca27108c > > > Diff: https://reviews.apache.org/r/64542/diff/1/ > > > Testing > --- > > - enabled kerberos, ran service checks on onefs > > > Thanks, > > Attila Magyar > >
Re: Review Request 64242: AMBARI-22476: Modify bootstrap API to pass Ambari repo URL from UI and retrieve Host OS types
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64242/#review193550 --- This has been pushed. Please close this review and associated JIRA. - Nate Cole On Dec. 1, 2017, 2:47 a.m., Sonia Garudi wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64242/ > --- > > (Updated Dec. 1, 2017, 2:47 a.m.) > > > Review request for Ambari and Nate Cole. > > > Bugs: AMBARI-22476 > https://issues.apache.org/jira/browse/AMBARI-22476 > > > Repository: ambari > > > Description > --- > > The hosts failing with error code 44 are bootstrapped along with the ambari > repository URLs. The required java beans are changed to include the ambari > repository URLs to be passed to bootstrap.py. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/bootstrap/BSRunner.java > c7976ee5bb48209478258625532e3420973b4b8c > > ambari-server/src/main/java/org/apache/ambari/server/bootstrap/SshHostInfo.java > 86888f499f15aa23807067ee1af745525b9d4619 > ambari-server/src/main/python/bootstrap.py > f1c53ce45316843fd38624536eed656b029a230a > > ambari-server/src/test/java/org/apache/ambari/server/bootstrap/BootStrapTest.java > b9da013524fee556c6faa2f0236d2c0c4b032750 > ambari-server/src/test/python/TestBootstrap.py > bea47f498cbb523fbe24622c63b48918d433c717 > ambari-web/app/controllers/wizard/step3_controller.js > 5ff2cdc1b4bb9da20565fe387765b4fc6728e096 > ambari-web/test/controllers/wizard/step3_test.js > be1affe28fdcdb3404ebd3fb3fc6e61315483ed2 > > > Diff: https://reviews.apache.org/r/64242/diff/1/ > > > Testing > --- > > Tests were successful in ambari-web and ambari-server. > > > Thanks, > > Sonia Garudi > >
Re: Review Request 64407: AMBARI-22592: Set ambari.repo. in ambari-server setup script
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64407/#review193549 --- This has been pushed. Please close this review and associated JIRA. - Nate Cole On Dec. 8, 2017, 5:57 a.m., Sneha Kanekar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64407/ > --- > > (Updated Dec. 8, 2017, 5:57 a.m.) > > > Review request for Ambari and Nate Cole. > > > Bugs: AMBARI-22592 > https://issues.apache.org/jira/browse/AMBARI-22592 > > > Repository: ambari > > > Description > --- > > The ambari-server setup script is updated to allow the user to store/update > Ambari repo base URL in ambari.properties file for each OS type through > command line. The command to set Ambari repo URL : ambari-server setup > --ambari-repo > > > Diffs > - > > ambari-server/src/main/python/ambari-server.py 188910e > ambari-server/src/main/python/ambari_server/serverConfiguration.py 27b4472 > ambari-server/src/main/python/ambari_server/serverSetup.py 3b41fff > ambari-server/src/test/python/TestAmbariServer.py ad4e371 > > > Diff: https://reviews.apache.org/r/64407/diff/2/ > > > Testing > --- > > Tests were successful in ambari-server. > > > Thanks, > > Sneha Kanekar > >
Re: Review Request 64242: AMBARI-22476: Modify bootstrap API to pass Ambari repo URL from UI and retrieve Host OS types
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64242/#review193548 --- Ship it! Will mark this as Ship It, but the repo url should be cleaned up, and (hopefully) doesn't need to be a json structure. - Nate Cole On Dec. 1, 2017, 2:47 a.m., Sonia Garudi wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64242/ > --- > > (Updated Dec. 1, 2017, 2:47 a.m.) > > > Review request for Ambari and Nate Cole. > > > Bugs: AMBARI-22476 > https://issues.apache.org/jira/browse/AMBARI-22476 > > > Repository: ambari > > > Description > --- > > The hosts failing with error code 44 are bootstrapped along with the ambari > repository URLs. The required java beans are changed to include the ambari > repository URLs to be passed to bootstrap.py. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/bootstrap/BSRunner.java > c7976ee5bb48209478258625532e3420973b4b8c > > ambari-server/src/main/java/org/apache/ambari/server/bootstrap/SshHostInfo.java > 86888f499f15aa23807067ee1af745525b9d4619 > ambari-server/src/main/python/bootstrap.py > f1c53ce45316843fd38624536eed656b029a230a > > ambari-server/src/test/java/org/apache/ambari/server/bootstrap/BootStrapTest.java > b9da013524fee556c6faa2f0236d2c0c4b032750 > ambari-server/src/test/python/TestBootstrap.py > bea47f498cbb523fbe24622c63b48918d433c717 > ambari-web/app/controllers/wizard/step3_controller.js > 5ff2cdc1b4bb9da20565fe387765b4fc6728e096 > ambari-web/test/controllers/wizard/step3_test.js > be1affe28fdcdb3404ebd3fb3fc6e61315483ed2 > > > Diff: https://reviews.apache.org/r/64242/diff/1/ > > > Testing > --- > > Tests were successful in ambari-web and ambari-server. > > > Thanks, > > Sonia Garudi > >
Re: Review Request 64407: AMBARI-22592: Set ambari.repo. in ambari-server setup script
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64407/#review193547 --- Ship it! Ship It! - Nate Cole On Dec. 8, 2017, 5:57 a.m., Sneha Kanekar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64407/ > --- > > (Updated Dec. 8, 2017, 5:57 a.m.) > > > Review request for Ambari and Nate Cole. > > > Bugs: AMBARI-22592 > https://issues.apache.org/jira/browse/AMBARI-22592 > > > Repository: ambari > > > Description > --- > > The ambari-server setup script is updated to allow the user to store/update > Ambari repo base URL in ambari.properties file for each OS type through > command line. The command to set Ambari repo URL : ambari-server setup > --ambari-repo > > > Diffs > - > > ambari-server/src/main/python/ambari-server.py 188910e > ambari-server/src/main/python/ambari_server/serverConfiguration.py 27b4472 > ambari-server/src/main/python/ambari_server/serverSetup.py 3b41fff > ambari-server/src/test/python/TestAmbariServer.py ad4e371 > > > Diff: https://reviews.apache.org/r/64407/diff/2/ > > > Testing > --- > > Tests were successful in ambari-server. > > > Thanks, > > Sneha Kanekar > >
Re: Review Request 64242: AMBARI-22476: Modify bootstrap API to pass Ambari repo URL from UI and retrieve Host OS types
> On Dec. 4, 2017, 11:20 a.m., Nate Cole wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/bootstrap/SshHostInfo.java
> > Lines 130-132 (patched)
> > <https://reviews.apache.org/r/64242/diff/1/?file=1905504#file1905504line130>
> >
> > How many URLs are we talking about here. If more than one, should this
> > bet a Set?
>
> Sonia Garudi wrote:
> ambariRepoUrls is a json string having ambari repo URLs for each os type
> which is sent from the UI.
>
> Nate Cole wrote:
> Can I see an example?
>
> Sonia Garudi wrote:
> Example :
>
> "[{"os_type":"os1","ambari_repo":"http://ambari-repo1","hosts":["host1","host2"],"ambariRepoUIError":"","hasError":false},{"os_type":"os2","ambari_repo":"http://ambari-repo2","hosts":["host3"],"ambariRepoUIError":"","hasError":false}];
> The above json string consists of list of 2 objects; each containing the
> os_type, ambari repository URL and the list of host names for the
> corresponding os_type. The 2 other fields(i.e. ambariRepoUIError and
> hasError) are used for UI validation.
>
> Nate Cole wrote:
> This seems strange in that we end up with two different ways to "get" the
> URL. One is that when the UI does not recognize an OS, we ask for it and
> (presumably) save it in ambari.properties. Seems as though when we should
> always get the value from ambari.properties and fail if it's not there.
> Also, why are we setting all this information If we have a command for
> host1, host2, why would we need to know that os-type-2 goes on host3?
>
> Sonia Garudi wrote:
> The bootstrap.py file first checks if the os type of a host is not same
> as server os type. If its different it checks for the ambari repo property
> for that os type in the ambari.properties file.
> case 1 : If the property is not found, the bootstrap for that host fails
> with code 44. The UI prompts for the ambari repo URL for os_type of hosts
> which fail with code 44. These hosts are bootstrapped with the ambariRepoUrls
> json string which contains the ambari repo urls. These URLs are then saved in
> the ambari.properties file for the respective os type.
> case 2 : If the property is found in the properties file, the
> installation will continue without showing a prompt for that os_type.
>
> The hosts field in each object is used for removal of prompt when a host
> is removed.
> Implementation of storing of ambari repo url in properties file in done
> in upcoming patches.
>
> Yussuf Shaikh wrote:
> Hi Nate,
>
> Sonia just dumped the data structure used on step3 controller to the
> bootstrap API, hence it contains the hostnames, but at server side only
> os_type and corresponding repo URL is used.
>
> We can just do away with hosts information in the json data which would
> look like below:
>
> [{"os_type":"os1","ambari_repo":"http://ambari-repo1"},{"os_type":"os2","ambari_repo":"http://ambari-repo2"}];
>
> Let us know if you really need to change this in this patch or we could
> change it in next patch (AMBARI-22394).
I was just looking for clarification of what's in this string. If it's just a
singular URL for the repo of that host, or what purpose the structure serves.
Think of it this way. With no UI interaction at all, what should the
bootstrapping do? Would we make people create that odd json structure and pass
it as a string? I'm ok with this patch as-is, but we should be careful what is
really being asked of the API here in the next patch.
- Nate
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64242/#review192705
---
On Dec. 1, 2017, 2:47 a.m., Sonia Garudi wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64242/
> ---
>
> (Updated Dec. 1, 2017, 2:47 a.m.)
>
>
> Review request for Ambari and Nate Cole.
>
>
> Bugs: AMBARI-22476
> https://issues.apache.org/jira/browse/AMBARI-22476
>
>
> Repository: ambari
>
>
> Description
> ---
>
> The hosts failing with error code 44 are bootstrapped along with the ambari
> repository URLs. The
Re: Review Request 64502: YARN Shuffle Service Can't Be Found On Client-Only Nodes After New Cluster Install
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64502/#review193416
---
Ship it!
Ship It!
- Nate Cole
On Dec. 11, 2017, 12:04 p.m., Jonathan Hurley wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64502/
> ---
>
> (Updated Dec. 11, 2017, 12:04 p.m.)
>
>
> Review request for Ambari, Dmitro Lisnichenko and Nate Cole.
>
>
> Bugs: AMBARI-22628
> https://issues.apache.org/jira/browse/AMBARI-22628
>
>
> Repository: ambari
>
>
> Description
> ---
>
> Installing a new cluster can create values in yarn-site.xml which have
> {{None}} specified in the classpath for Spark
>
> ```
>
> yarn.nodemanager.aux-services.spark2_shuffle.classpath
> /usr/hdp/None/spark2/aux/*
>
>
>
> yarn.nodemanager.aux-services.spark_shuffle.classpath
> /usr/hdp/None/spark/aux/*
>
>
>
>
> yarn.timeline-service.entity-group-fs-store.group-id-plugin-classpath
> /usr/hdp/None/spark/hdpLib/*
>
> ```
>
> The cause for this is that YARN Clients on hosts without daemons never get a
> restart command after the initial {{yarn-site.xml}}, and can never fill in
> the correct values. This causes problems when jobs are run on these nodes:
>
> ```
> 2017-12-04 10:16:41,789 INFO service.AbstractService
> (AbstractService.java:noteFailure(272)) - Service
> org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices failed
> in state INITED; cause: java.lang.ClassNotFoundException:
> org.apache.spark.network.yarn.YarnShuffleService
> java.lang.ClassNotFoundException:
> org.apache.spark.network.yarn.YarnShuffleService
> ```
>
>
> Diffs
> -
>
>
> ambari-server/src/main/java/org/apache/ambari/server/agent/ExecutionCommand.java
> 9d5e29ee8a
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClientConfigResourceProvider.java
> a7c712bd1a
> ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java
> 15efcd2173
>
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
> ce328f91ff
>
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py
> 98141456c7
>
>
> Diff: https://reviews.apache.org/r/64502/diff/3/
>
>
> Testing
> ---
>
> Manual install via UI and Blueprint
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 64419: Hive Queries Failing with Missing SnappyCodec Compression Type Due to Hive's Use of MapReduce Admin Env Property
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64419/#review193165
---
Ship it!
Ship It!
- Nate Cole
On Dec. 7, 2017, 3:53 p.m., Jonathan Hurley wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64419/
> ---
>
> (Updated Dec. 7, 2017, 3:53 p.m.)
>
>
> Review request for Ambari, Dmitro Lisnichenko and Nate Cole.
>
>
> Bugs: AMBARI-22613
> https://issues.apache.org/jira/browse/AMBARI-22613
>
>
> Repository: ambari
>
>
> Description
> ---
>
> Many queries, similar to the below {{InsertOverwrite}}, are failing after
> performing an HDP->HDP upgrade. The cause is that Hive does not localize the
> Tez TAR file, but instead uses the MapReduce2 property
> {{mapreduce.admin.user.env}}
>
> ```
> >>> analyze table studenttab10k compute statistics for columns age ;
> INFO : Session is already open
> INFO : Dag name: analyze table studenttab10k compute st...age(Stage-0)
> INFO : Status: Running (Executing on YARN cluster with App id
> application_1512568339263_0003)
>
> [2K
> [2K[36;1mVERTICES STATUS TOTAL COMPLETED RUNNING PENDING
> FAILED KILLED
> [22;0m[2K
> [2KMap 1RUNNING 1 010 2
> 0
> Reducer 2 INITED 1 001 0
> 0
> [2K
> [2K[31;1mVERTICES: 00/02 [>>--] 0%ELAPSED
> TIME: 9.38 s
> [22;0m[2K
> [8A[2K
> [2K[36;1mVERTICES STATUS TOTAL COMPLETED RUNNING PENDING
> FAILED KILLED
> [22;0m[2K
> [2KMap 1RUNNING 1 001 4
> 0
> Reducer 2 INITED 1 001 0
> 0
> [2K
> [2K[31;1mVERTICES: 00/02 [>>--] 0%ELAPSED
> TIME: 14.37 s
> [22;0m[2K
> ERROR : Status: Failed
> ERROR : Vertex failed, vertexName=Map 1,
> vertexId=vertex_1512568339263_0003_2_00, diagnostics=[Task failed,
> taskId=task_1512568339263_0003_2_00_00, diagnostics=[TaskAttempt 0
> failed, info=[Error: Failure while running task:java.lang.RuntimeException:
> java.io.IOException: Unable to get CompressorType for codec
> (org.apache.hadoop.io.compress.SnappyCodec). This is most likely due to
> missing native libraries for the codec.
> Caused by: java.lang.RuntimeException: native snappy library not available:
> this version of libhadoop was built without snappy support.
> at
> org.apache.hadoop.io.compress.SnappyCodec.checkNativeCodeLoaded(SnappyCodec.java:65)
> at
> org.apache.hadoop.io.compress.SnappyCodec.getCompressorType(SnappyCodec.java:134)
> at
> org.apache.tez.runtime.library.common.sort.impl.ExternalSorter.(ExternalSorter.java:208)
> ... 18 more
> ], TaskAttempt 1 failed, info=[Error: Failure while running
> task:java.lang.RuntimeException: java.io.IOException: Unable to get
> CompressorType for codec (org.apache.hadoop.io.compress.SnappyCodec). This is
> most likely due to missing native libraries for the codec.
> ```
>
>
> Diffs
> -
>
>
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py
> 18e297865e
> ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/config-upgrade.xml
> 6e1c81968d
>
>
> Diff: https://reviews.apache.org/r/64419/diff/2/
>
>
> Testing
> ---
>
> Manual upgrade testing.
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 64419: Hive Queries Failing with Missing SnappyCodec Compression Type Due to Hive's Use of MapReduce Admin Env Property
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64419/#review193135
---
Ship it!
Ship It!
- Nate Cole
On Dec. 7, 2017, 11:55 a.m., Jonathan Hurley wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64419/
> ---
>
> (Updated Dec. 7, 2017, 11:55 a.m.)
>
>
> Review request for Ambari, Dmitro Lisnichenko and Nate Cole.
>
>
> Bugs: AMBARI-22613
> https://issues.apache.org/jira/browse/AMBARI-22613
>
>
> Repository: ambari
>
>
> Description
> ---
>
> Many queries, similar to the below {{InsertOverwrite}}, are failing after
> performing an HDP->HDP upgrade. The cause is that Hive does not localize the
> Tez TAR file, but instead uses the MapReduce2 property
> {{mapreduce.admin.user.env}}
>
> ```
> >>> analyze table studenttab10k compute statistics for columns age ;
> INFO : Session is already open
> INFO : Dag name: analyze table studenttab10k compute st...age(Stage-0)
> INFO : Status: Running (Executing on YARN cluster with App id
> application_1512568339263_0003)
>
> [2K
> [2K[36;1mVERTICES STATUS TOTAL COMPLETED RUNNING PENDING
> FAILED KILLED
> [22;0m[2K
> [2KMap 1RUNNING 1 010 2
> 0
> Reducer 2 INITED 1 001 0
> 0
> [2K
> [2K[31;1mVERTICES: 00/02 [>>--] 0%ELAPSED
> TIME: 9.38 s
> [22;0m[2K
> [8A[2K
> [2K[36;1mVERTICES STATUS TOTAL COMPLETED RUNNING PENDING
> FAILED KILLED
> [22;0m[2K
> [2KMap 1RUNNING 1 001 4
> 0
> Reducer 2 INITED 1 001 0
> 0
> [2K
> [2K[31;1mVERTICES: 00/02 [>>--] 0%ELAPSED
> TIME: 14.37 s
> [22;0m[2K
> ERROR : Status: Failed
> ERROR : Vertex failed, vertexName=Map 1,
> vertexId=vertex_1512568339263_0003_2_00, diagnostics=[Task failed,
> taskId=task_1512568339263_0003_2_00_00, diagnostics=[TaskAttempt 0
> failed, info=[Error: Failure while running task:java.lang.RuntimeException:
> java.io.IOException: Unable to get CompressorType for codec
> (org.apache.hadoop.io.compress.SnappyCodec). This is most likely due to
> missing native libraries for the codec.
> Caused by: java.lang.RuntimeException: native snappy library not available:
> this version of libhadoop was built without snappy support.
> at
> org.apache.hadoop.io.compress.SnappyCodec.checkNativeCodeLoaded(SnappyCodec.java:65)
> at
> org.apache.hadoop.io.compress.SnappyCodec.getCompressorType(SnappyCodec.java:134)
> at
> org.apache.tez.runtime.library.common.sort.impl.ExternalSorter.(ExternalSorter.java:208)
> ... 18 more
> ], TaskAttempt 1 failed, info=[Error: Failure while running
> task:java.lang.RuntimeException: java.io.IOException: Unable to get
> CompressorType for codec (org.apache.hadoop.io.compress.SnappyCodec). This is
> most likely due to missing native libraries for the codec.
> ```
>
>
> Diffs
> -
>
>
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/configuration-mapred/mapred-site.xml
> 398c9d7d20
>
> ambari-server/src/main/resources/stacks/HDP/2.2/services/YARN/configuration-mapred/mapred-site.xml
> 099e38866e
> ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/config-upgrade.xml
> 6e1c81968d
>
>
> Diff: https://reviews.apache.org/r/64419/diff/1/
>
>
> Testing
> ---
>
> Manual upgrade testing.
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 64242: AMBARI-22476: Modify bootstrap API to pass Ambari repo URL from UI and retrieve Host OS types
> On Dec. 4, 2017, 11:20 a.m., Nate Cole wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/bootstrap/SshHostInfo.java
> > Lines 130-132 (patched)
> > <https://reviews.apache.org/r/64242/diff/1/?file=1905504#file1905504line130>
> >
> > How many URLs are we talking about here. If more than one, should this
> > bet a Set?
>
> Sonia Garudi wrote:
> ambariRepoUrls is a json string having ambari repo URLs for each os type
> which is sent from the UI.
>
> Nate Cole wrote:
> Can I see an example?
>
> Sonia Garudi wrote:
> Example :
>
> "[{"os_type":"os1","ambari_repo":"http://ambari-repo1","hosts":["host1","host2"],"ambariRepoUIError":"","hasError":false},{"os_type":"os2","ambari_repo":"http://ambari-repo2","hosts":["host3"],"ambariRepoUIError":"","hasError":false}];
> The above json string consists of list of 2 objects; each containing the
> os_type, ambari repository URL and the list of host names for the
> corresponding os_type. The 2 other fields(i.e. ambariRepoUIError and
> hasError) are used for UI validation.
This seems strange in that we end up with two different ways to "get" the URL.
One is that when the UI does not recognize an OS, we ask for it and
(presumably) save it in ambari.properties. Seems as though when we should
always get the value from ambari.properties and fail if it's not there. Also,
why are we setting all this information If we have a command for host1, host2,
why would we need to know that os-type-2 goes on host3?
- Nate
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64242/#review192705
---
On Dec. 1, 2017, 2:47 a.m., Sonia Garudi wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64242/
> ---
>
> (Updated Dec. 1, 2017, 2:47 a.m.)
>
>
> Review request for Ambari and Nate Cole.
>
>
> Bugs: AMBARI-22476
> https://issues.apache.org/jira/browse/AMBARI-22476
>
>
> Repository: ambari
>
>
> Description
> ---
>
> The hosts failing with error code 44 are bootstrapped along with the ambari
> repository URLs. The required java beans are changed to include the ambari
> repository URLs to be passed to bootstrap.py.
>
>
> Diffs
> -
>
>
> ambari-server/src/main/java/org/apache/ambari/server/bootstrap/BSRunner.java
> c7976ee5bb48209478258625532e3420973b4b8c
>
> ambari-server/src/main/java/org/apache/ambari/server/bootstrap/SshHostInfo.java
> 86888f499f15aa23807067ee1af745525b9d4619
> ambari-server/src/main/python/bootstrap.py
> f1c53ce45316843fd38624536eed656b029a230a
>
> ambari-server/src/test/java/org/apache/ambari/server/bootstrap/BootStrapTest.java
> b9da013524fee556c6faa2f0236d2c0c4b032750
> ambari-server/src/test/python/TestBootstrap.py
> bea47f498cbb523fbe24622c63b48918d433c717
> ambari-web/app/controllers/wizard/step3_controller.js
> 5ff2cdc1b4bb9da20565fe387765b4fc6728e096
> ambari-web/test/controllers/wizard/step3_test.js
> be1affe28fdcdb3404ebd3fb3fc6e61315483ed2
>
>
> Diff: https://reviews.apache.org/r/64242/diff/1/
>
>
> Testing
> ---
>
> Tests were successful in ambari-web and ambari-server.
>
>
> Thanks,
>
> Sonia Garudi
>
>
Re: Review Request 64407: AMBARI-22592: Set ambari.repo. in ambari-server setup script
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64407/#review193119 --- ambari-server/src/main/python/ambari_server/serverSetup.py Lines 1140-1141 (patched) <https://reviews.apache.org/r/64407/#comment271623> Can this somehow be validated to the known, good os family values? If not, this is still ok. ambari-server/src/main/python/ambari_server/serverSetup.py Lines 1199 (patched) <https://reviews.apache.org/r/64407/#comment271622> nit: formatting (odd space) - Nate Cole On Dec. 7, 2017, 4:35 a.m., Sneha Kanekar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64407/ > --- > > (Updated Dec. 7, 2017, 4:35 a.m.) > > > Review request for Ambari and Nate Cole. > > > Bugs: AMBARI-22592 > https://issues.apache.org/jira/browse/AMBARI-22592 > > > Repository: ambari > > > Description > --- > > The ambari-server setup script is updated to allow the user to store/update > Ambari repo base URL in ambari.properties file for each OS type through > command line. The command to set Ambari repo URL : ambari-server setup > --ambari-repo > > > Diffs > - > > ambari-server/src/main/python/ambari-server.py 188910e > ambari-server/src/main/python/ambari_server/serverConfiguration.py f744fa0 > ambari-server/src/main/python/ambari_server/serverSetup.py af45584 > ambari-server/src/test/python/TestAmbariServer.py ad4e371 > > > Diff: https://reviews.apache.org/r/64407/diff/1/ > > > Testing > --- > > Tests were successful in ambari-server. > > > Thanks, > > Sneha Kanekar > >
Re: Review Request 64242: AMBARI-22476: Modify bootstrap API to pass Ambari repo URL from UI and retrieve Host OS types
> On Dec. 4, 2017, 11:20 a.m., Nate Cole wrote: > > ambari-server/src/main/java/org/apache/ambari/server/bootstrap/SshHostInfo.java > > Lines 130-132 (patched) > > <https://reviews.apache.org/r/64242/diff/1/?file=1905504#file1905504line130> > > > > How many URLs are we talking about here. If more than one, should this > > bet a Set? > > Sonia Garudi wrote: > ambariRepoUrls is a json string having ambari repo URLs for each os type > which is sent from the UI. Can I see an example? - Nate --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64242/#review192705 --- On Dec. 1, 2017, 2:47 a.m., Sonia Garudi wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64242/ > --- > > (Updated Dec. 1, 2017, 2:47 a.m.) > > > Review request for Ambari and Nate Cole. > > > Bugs: AMBARI-22476 > https://issues.apache.org/jira/browse/AMBARI-22476 > > > Repository: ambari > > > Description > --- > > The hosts failing with error code 44 are bootstrapped along with the ambari > repository URLs. The required java beans are changed to include the ambari > repository URLs to be passed to bootstrap.py. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/bootstrap/BSRunner.java > c7976ee5bb48209478258625532e3420973b4b8c > > ambari-server/src/main/java/org/apache/ambari/server/bootstrap/SshHostInfo.java > 86888f499f15aa23807067ee1af745525b9d4619 > ambari-server/src/main/python/bootstrap.py > f1c53ce45316843fd38624536eed656b029a230a > > ambari-server/src/test/java/org/apache/ambari/server/bootstrap/BootStrapTest.java > b9da013524fee556c6faa2f0236d2c0c4b032750 > ambari-server/src/test/python/TestBootstrap.py > bea47f498cbb523fbe24622c63b48918d433c717 > ambari-web/app/controllers/wizard/step3_controller.js > 5ff2cdc1b4bb9da20565fe387765b4fc6728e096 > ambari-web/test/controllers/wizard/step3_test.js > be1affe28fdcdb3404ebd3fb3fc6e61315483ed2 > > > Diff: https://reviews.apache.org/r/64242/diff/1/ > > > Testing > --- > > Tests were successful in ambari-web and ambari-server. > > > Thanks, > > Sonia Garudi > >
Re: Review Request 64340: PREVIEW - Livy server start fails during EU with 'Address already in use' error
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64340/#review192871
---
Ship it!
Ship It!
- Nate Cole
On Dec. 5, 2017, 10:04 a.m., Dmitro Lisnichenko wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64340/
> ---
>
> (Updated Dec. 5, 2017, 10:04 a.m.)
>
>
> Review request for Ambari, Dmytro Grinenko, Jonathan Hurley, and Nate Cole.
>
>
> Bugs: AMBARI-22594
> https://issues.apache.org/jira/browse/AMBARI-22594
>
>
> Repository: ambari
>
>
> Description
> ---
>
> Observed this issue quite consistently in Ambari-2.6.1 Upgrade ST runs
>
> *STR*
> # Deployed cluster with Ambari version: 2.5.1.0-159 and HDP version:
> 2.6.1.0-129
> # Upgrade Ambari to Target Version: 2.6.1.0-43 | Hash:
> acbce28fdd119c72625c6beff63fc169de58ba22
> # Regenerate keytabs post Ambari upgrade and this step will restart all
> services. Here Livy server is operational and gets restarted fine (at
> timestamp: 09:29)
> # Now register HDP-2.6.4.0-36 version and perform EU. During EU 'Restart Livy
> server' task happens and reports success (at timestamp: 10:26)
> # However when checking the livy logs - Livy restart reported below exception
> as the previous process was not killed/stopped
> {code}
> 17/11/21 10:26:22 WARN AbstractLifeCycle: FAILED
> org.eclipse.jetty.server.Server@3bc735b3: java.net.BindException: Address
> already in use
> java.net.BindException: Address already in use
> at sun.nio.ch.Net.bind0(Native Method)
> at sun.nio.ch.Net.bind(Net.java:433)
> at sun.nio.ch.Net.bind(Net.java:425)
> at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
> at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
> at org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:321)
> at org.apache.livy.server.LivyServer.main(LivyServer.scala)
> Exception in thread "main" java.net.BindException: Address already in use
> at sun.nio.ch.Net.bind0(Native Method)
> at sun.nio.ch.Net.bind(Net.java:433)
> at sun.nio.ch.Net.bind(Net.java:425)
> at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
> {code}
>
> - Post Upgrade, I tried to stop/start Spark as well and Livy still gave same
> exception; although web Ui reports operation as success (at timestamp: 11:37)
> - Finally the web UI shows Livy as down, even though the process is running
> from the initial step (at timestamp: 09:29)
>
>
> Diffs
> -
>
>
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/livy_service.py
> a78f50c077
>
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/params.py
> 9b813a13f0
>
>
> Diff: https://reviews.apache.org/r/64340/diff/1/
>
>
> Testing
> ---
>
> waiting for feedback from reviewers to apply this change to all Livy and
> Livy2 service scripts
>
>
> Thanks,
>
> Dmitro Lisnichenko
>
>
Re: Review Request 64316: Messages for some services during PU package installation indicate circular dependency
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64316/#review192866 --- Ship it! Ship It! - Nate Cole On Dec. 4, 2017, 5:13 p.m., Jonathan Hurley wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64316/ > --- > > (Updated Dec. 4, 2017, 5:13 p.m.) > > > Review request for Ambari, Dmitro Lisnichenko and Nate Cole. > > > Bugs: AMBARI-22590 > https://issues.apache.org/jira/browse/AMBARI-22590 > > > Repository: ambari > > > Description > --- > > Written this as 'Blocker' severity, please triage once and decide > > This issue could affect user experience for Patch Upgrades > > *Case 1:* Register Patch VDF for Mapreduce service > Try to install the packages > Result: Error for missing Hive service in the VDF (see MR-only.png) > > *Case 2:* Add Hive to the patch VDF, os that it now includes both MR and Hive > Try to install the packages > Result: Error for missing Tez service in the VDF (see MR-and-Hive) > > This is now confusing as the dependency looks cyclic, we asked to add Hive in > first case and now asking to add MR again > > *Case 3:* Add a new VDF for only Mahout or Oozie. This would prompt for MR > due to dependency. Now you need to add MR and Hive and then Tez too (see > Mahout-only.png) > > *Expected result:* The message for dependent services should recursively > decide the services and print them all at once. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/checks/RequiredServicesInRepositoryCheck.java > d9114113b2 > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProvider.java > d155d46a58 > > ambari-server/src/main/java/org/apache/ambari/server/state/repository/VersionDefinitionXml.java > 69e22b37e0 > > ambari-server/src/test/java/org/apache/ambari/server/checks/RequiredServicesInRepositoryCheckTest.java > 59848719ad > > ambari-server/src/test/java/org/apache/ambari/server/state/repository/VersionDefinitionTest.java > 9fe6146a9c > > > Diff: https://reviews.apache.org/r/64316/diff/2/ > > > Testing > --- > > PENDING > > > Thanks, > > Jonathan Hurley > >
Review Request 64310: Storm service check failed during PU due to CNF StormAtlasHook
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64310/
---
Review request for Ambari and Jonathan Hurley.
Bugs: AMBARI-22587
https://issues.apache.org/jira/browse/AMBARI-22587
Repository: ambari
Description
---
* Installation of {{storm_\{version\}}} packages triggers the install of
{{atlas-metadata_\{version\}-storm-plugin}} package.
* When upgrading including ATLAS but not STORM, the atlas bits are put down
just fine, but {{/usr/hdp/\{new_version\}/atlas/hook/storm}} doesn't exist.
* Symlinks of {{storm/extlib}} to {{atlas/hook/storm}} become broken because
the hooks aren't there.
We should be requiring a patch of ATLAS to also require STORM be patched as
well in order to get the plugins installed.
Diffs
-
ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_packages.json
1783655e8e
Diff: https://reviews.apache.org/r/64310/diff/1/
Testing
---
Manual.
Thanks,
Nate Cole
Re: Review Request 64308: Remove Beacon conf-select Changes Since It's Not Part of HDP
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64308/#review192739
---
Ship it!
Ship It!
- Nate Cole
On Dec. 4, 2017, 12:57 p.m., Jonathan Hurley wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64308/
> ---
>
> (Updated Dec. 4, 2017, 12:57 p.m.)
>
>
> Review request for Ambari and Nate Cole.
>
>
> Bugs: AMBARI-22586
> https://issues.apache.org/jira/browse/AMBARI-22586
>
>
> Repository: ambari
>
>
> Description
> ---
>
> Install Beacon (DLM) + HDP:
>
> {code}
> [root@ctr-e135-1512069032975-1130-01-05 ~]# ls /etc/beacon/conf
> beacon-env.jceks beacon-security-site.jceks beacon-security-site.xml
> beacon.yml
> {code}
>
> {code}
> [root@ctr-e135-1512069032975-1130-01-05 ~]# ls /usr/dlm/current/beacon/
> bin client conf data libext log logs pids run server sql webapp
> [root@ctr-e135-1512069032975-1130-01-05 ~]# ls -lah
> /usr/dlm/current/beacon/
> total 36K
> drwxr-xr-x 9 beacon root 4.0K Dec 1 11:38 .
> drwxr-xr-x 4 root root 4.0K Dec 1 11:38 ..
> drwxr-xr-x 2 root root 4.0K Dec 1 11:38 bin
> drwxr-xr-x 3 root root 4.0K Dec 1 11:38 client
> lrwxrwxrwx 1 root root 16 Dec 1 11:38 conf -> /etc/beacon/conf
> drwxr-xr-x 2 beacon beacon 4.0K Dec 1 11:38 data
> drwxr-xr-x 2 beacon beacon 4.0K Oct 30 20:57 libext
> lrwxrwxrwx 1 root root 15 Dec 1 11:38 log -> /var/log/beacon
> lrwxrwxrwx 1 beacon beacon 15 Dec 1 11:38 logs -> /var/log/beacon
> lrwxrwxrwx 1 beacon beacon 15 Dec 1 11:38 pids -> /var/run/beacon
> lrwxrwxrwx 1 root root 15 Dec 1 11:38 run -> /var/run/beacon
> drwxr-xr-x 2 root root 4.0K Dec 1 11:38 server
> drwxr-xr-x 2 beacon beacon 4.0K Dec 1 11:38 sql
> drwxr-xr-x 2 beacon beacon 4.0K Dec 1 11:38 webapp
> [root@ctr-e135-1512069032975-1130-01-05 ~]# ls -lah /etc/beacon/
> total 20K
> drwxr-xr-x 3 root root 4.0K Dec 1 11:38 .
> drwxr-xr-x 125 root root 12K Dec 1 12:37 ..
> lrwxrwxrwx 1 root root 35 Dec 1 11:38 conf ->
> /usr/hdp/current/beacon-client/conf
> drwxr-xr-x 3 root root 4.0K Dec 1 11:38 conf.backup
> [root@ctr-e135-1512069032975-1130-01-05 ~]#
> {code}
>
> It looks like it was accidentally added to HDP's {{stack_packages.json}}
> since {{hdp-select}} supported it for a brief time:
> {code}
> [root@c6402 ~]# hdp-select status | grep beacon
> beacon - None
> beacon-client - None
> beacon-server - None
> {code}
>
>
> Diffs
> -
>
>
> ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_packages.json
> 1783655e8e
>
>
> Diff: https://reviews.apache.org/r/64308/diff/1/
>
>
> Testing
> ---
>
> Manual
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 64261: Update LDAP Authentication process to work with improved user management facility
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64261/#review192707 --- Ship it! ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java Lines 701-702 (original), 701-702 (patched) <https://reviews.apache.org/r/64261/#comment270959> This property seems important enough to add @Markdown - Nate Cole On Dec. 1, 2017, 4:53 p.m., Robert Levas wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64261/ > --- > > (Updated Dec. 1, 2017, 4:53 p.m.) > > > Review request for Ambari, Attila Magyar, Balázs Bence Sári, Eugene > Chekanskiy, Jonathan Hurley, Nate Cole, Robert Nettleton, and Sandor Molnar. > > > Bugs: AMBARI-21219 > https://issues.apache.org/jira/browse/AMBARI-21219 > > > Repository: ambari > > > Description > --- > > Update LDAP Authentication process to work with improved user management > facility > > > Diffs > - > > ambari-server/docs/configuration/index.md 514e9edcc3 > > ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java > 8904199cfd > > ambari-server/src/main/java/org/apache/ambari/server/ldap/service/AmbariLdapConfigurationProvider.java > f1e1881cea > ambari-server/src/main/java/org/apache/ambari/server/orm/dao/UserDAO.java > 0e28e50709 > > ambari-server/src/main/java/org/apache/ambari/server/security/authentication/AmbariAuthenticationProvider.java > 0e5c913baa > > ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AmbariAuthentication.java > bf30b83dd1 > > ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AmbariLdapAuthenticationProvider.java > a6f4387398 > > ambari-server/src/main/java/org/apache/ambari/server/security/authorization/LdapServerProperties.java > a4a95165e1 > > ambari-server/src/main/java/org/apache/ambari/server/security/authorization/Users.java > ebe26a237d > > ambari-server/src/test/java/org/apache/ambari/server/configuration/ConfigurationTest.java > fdc4a2f1c3 > > ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariAuthenticationTest.java > 64ec2019f6 > > ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariLdapAuthenticationProviderForDuplicateUserTest.java > d465c019f7 > > ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariLdapAuthenticationProviderTest.java > 1aafaa1889 > > ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AuthorizationHelperTest.java > cad734c682 > > ambari-server/src/test/java/org/apache/ambari/server/security/authorization/TestUsers.java > e99bdfd5c6 > > > Diff: https://reviews.apache.org/r/64261/diff/2/ > > > Testing > --- > > Manually tested against OpenLDAP and Active Directory > > # Local test results: > ``` > [INFO] > > [INFO] BUILD SUCCESS > [INFO] > > [INFO] Total time: 30:23 min > [INFO] Finished at: 2017-12-01T16:35:50-05:00 > [INFO] Final Memory: 99M/2005M > [INFO] > > ``` > > # Jenkins test results: PENDING > > > Thanks, > > Robert Levas > >
Re: Review Request 64204: zkfc show no alerts when zkfc is down
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64204/#review192706 --- ambari-web/app/mappers/alert_definition_summary_mapper.js Lines 94-106 (patched) <https://reviews.apache.org/r/64204/#comment270958> Strange that this isn't getting covered in the above block. I'm not a UI engineer, but seems like special casing ZKFC isn't a good idea and that any SLAVE-type should be presenting it's alert data? - Nate Cole On Nov. 30, 2017, 6:40 a.m., zhangxiaolu zhangxiaolu wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64204/ > --- > > (Updated Nov. 30, 2017, 6:40 a.m.) > > > Review request for Ambari, Andrii Babiichuk, Alejandro Fernandez, Denys > Buzhor, Dmitro Lisnichenko, Dmytro Sen, Jaimin Jetly, Nate Cole, Richard > Zang, Srimanth Gunturi, Sid Wagle, Vitalyi Brodetskyi, and Yusaku Sako. > > > Bugs: AMBARI-22557 > https://issues.apache.org/jira/browse/AMBARI-22557 > > > Repository: ambari > > > Description > --- > > (1)zkfc show no alerts when zkfc is down > (2)we should deal with zkfc in front. > > > Diffs > - > > ambari-web/app/mappers/alert_definition_summary_mapper.js 29f286e > > > Diff: https://reviews.apache.org/r/64204/diff/1/ > > > Testing > --- > > done it > > > File Attachments > > > AMBARI-22557.patch > > https://reviews.apache.org/media/uploaded/files/2017/11/30/77e10ef5-6f44-4e18-9c8f-20d9fe62d8a9__AMBARI-22557.patch > > > Thanks, > > zhangxiaolu zhangxiaolu > >
Re: Review Request 64242: AMBARI-22476: Modify bootstrap API to pass Ambari repo URL from UI and retrieve Host OS types
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64242/#review192705 --- ambari-server/src/main/java/org/apache/ambari/server/bootstrap/SshHostInfo.java Lines 130-132 (patched) <https://reviews.apache.org/r/64242/#comment270956> How many URLs are we talking about here. If more than one, should this bet a Set? ambari-server/src/main/python/bootstrap.py Lines 906-908 (original), 908-910 (patched) <https://reviews.apache.org/r/64242/#comment270957> Then what? What is SharedState doing with this information/how is it processed? - Nate Cole On Dec. 1, 2017, 2:47 a.m., Sonia Garudi wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64242/ > --- > > (Updated Dec. 1, 2017, 2:47 a.m.) > > > Review request for Ambari and Nate Cole. > > > Bugs: AMBARI-22476 > https://issues.apache.org/jira/browse/AMBARI-22476 > > > Repository: ambari > > > Description > --- > > The hosts failing with error code 44 are bootstrapped along with the ambari > repository URLs. The required java beans are changed to include the ambari > repository URLs to be passed to bootstrap.py. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/bootstrap/BSRunner.java > c7976ee5bb48209478258625532e3420973b4b8c > > ambari-server/src/main/java/org/apache/ambari/server/bootstrap/SshHostInfo.java > 86888f499f15aa23807067ee1af745525b9d4619 > ambari-server/src/main/python/bootstrap.py > f1c53ce45316843fd38624536eed656b029a230a > > ambari-server/src/test/java/org/apache/ambari/server/bootstrap/BootStrapTest.java > b9da013524fee556c6faa2f0236d2c0c4b032750 > ambari-server/src/test/python/TestBootstrap.py > bea47f498cbb523fbe24622c63b48918d433c717 > ambari-web/app/controllers/wizard/step3_controller.js > 5ff2cdc1b4bb9da20565fe387765b4fc6728e096 > ambari-web/test/controllers/wizard/step3_test.js > be1affe28fdcdb3404ebd3fb3fc6e61315483ed2 > > > Diff: https://reviews.apache.org/r/64242/diff/1/ > > > Testing > --- > > Tests were successful in ambari-web and ambari-server. > > > Thanks, > > Sonia Garudi > >
Re: Review Request 63516: Remove properties.json And Switch To Adding Properties to ResourceProviders Dynamically
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63516/#review192704
---
Ship it!
Ship It!
- Nate Cole
On Dec. 4, 2017, 10:30 a.m., Dmitro Lisnichenko wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63516/
> ---
>
> (Updated Dec. 4, 2017, 10:30 a.m.)
>
>
> Review request for Ambari, Jonathan Hurley and Nate Cole.
>
>
> Bugs: AMBARI-22353
> https://issues.apache.org/jira/browse/AMBARI-22353
>
>
> Repository: ambari
>
>
> Description
> ---
>
> Legacy/ancient ResourceProviders use the {{properties.json}} file to govern
> which properties can be used with the provider. This seems like excessive
> decoupling without any benefit and usually leads to runtime errors when new
> or removed properties are forgotten.
>
> This file should be removed and the providers should be registering the known
> properties on their own.
>
>
> Diffs
> -
>
>
> ambari-logsearch/ambari-logsearch-server/src/test/java/org/apache/ambari/logsearch/web/security/LogsearchExternalServerAuthenticationProviderTest.java
> b6464c2d89
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/ResourceProviderFactory.java
> f6ca16bc69
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractAuthorizedResourceProvider.java
> a24c6394ac
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractControllerResourceProvider.java
> cc2548cc61
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractDRResourceProvider.java
> 953a2a7c77
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ActionResourceProvider.java
> 290931555e
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ActiveWidgetLayoutResourceProvider.java
> 389f0b2bf2
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AlertDefinitionResourceProvider.java
> 9579dca47d
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AlertGroupResourceProvider.java
> 0898bc3bee
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AlertHistoryResourceProvider.java
> 8d716eabe0
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AlertNoticeResourceProvider.java
> 8ffd596cfa
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AlertResourceProvider.java
> 4ab78b7ba6
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AlertTargetResourceProvider.java
> c1e9eb42a0
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AmbariPrivilegeResourceProvider.java
> 0ffceca84f
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintResourceProvider.java
> 8f4d62e032
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClientConfigResourceProvider.java
> f0ede5ddea
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterKerberosDescriptorResourceProvider.java
> 2678d55975
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterPrivilegeResourceProvider.java
> 60cf783940
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterResourceProvider.java
> b9b3b5668c
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProvider.java
> 1c36c96cc0
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ConfigGroupResourceProvider.java
> 737bfa4d46
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ConfigurationResourceProvider.java
> 0f260854f8
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/CredentialResourceProvider.java
> ebc7645828
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/DefaultProviderModule.java
> c3758b3f5e
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ExtensionLinkResourceProvider.java
> db904bf714
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ExtensionResourceProvider.java
> 53e44e1888
>
> ambari-server/src/main/java/org/ap
Re: Review Request 64205: Snapshot HBase task failed during IOP migration with TypeError
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64205/#review192370
---
Ship it!
Ship It!
- Nate Cole
On Nov. 30, 2017, 4:20 p.m., Dmitro Lisnichenko wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64205/
> ---
>
> (Updated Nov. 30, 2017, 4:20 p.m.)
>
>
> Review request for Ambari, Dmytro Grinenko, Jonathan Hurley, and Nate Cole.
>
>
> Bugs: AMBARI-22558
> https://issues.apache.org/jira/browse/AMBARI-22558
>
>
> Repository: ambari
>
>
> Description
> ---
>
> *STR*
> # Deployed cluster with Ambari version: 2.2.0 and IOP version: 4.2.0.0
> # Upgrade Ambari to Target Version: 2.5.2.0-298 | Hash:
> 2453e16418fd964042452b649153dbe45f3c6009
> # Upgrade Ambari to Target Version: 2.6.1.0-64 | Hash:
> cd4db8e9ac0ea7ce14fc1253959a121688f34952
> # Register HDP-2.6.4.0-51 and call remove iop-select
> # Install the new HDP version bits and start Express Upgrade
>
> *Result*
> Snapshot HBase task failed with below error:
> {code}
> Traceback (most recent call last):
> File
> "/var/lib/ambari-agent/cache/custom_actions/scripts/ru_execute_tasks.py",
> line 157, in
> ExecuteUpgradeTasks().execute()
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
> line 367, in execute
> method(env)
> File
> "/var/lib/ambari-agent/cache/custom_actions/scripts/ru_execute_tasks.py",
> line 153, in actionexecute
> shell.checked_call(task.command, logoutput=True, quiet=True)
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
> line 72, in inner
> result = function(command, **kwargs)
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
> line 102, in checked_call
> tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy)
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
> line 150, in _call_wrapper
> result = _call(command, **kwargs_copy)
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
> line 303, in _call
> raise ExecutionFailed(err_msg, code, out, err)
> resource_management.core.exceptions.ExecutionFailed: Execution of 'source
> /var/lib/ambari-agent/ambari-env.sh ; /usr/bin/ambari-python-wrap
> /var/lib/ambari-agent/cache/stacks/BigInsights/4.2/services/HBASE/package/scripts/hbase_upgrade.py
> take_snapshot /var/lib/ambari-agent/data/command-404.json
> /var/lib/ambari-agent/cache/custom_actions
> /var/lib/ambari-agent/data/structured-out-404.json INFO
> /var/lib/ambari-agent/tmp' returned 1. 2017-11-29 07:01:50,049 - Stack
> Feature Version Info: Cluster Stack=2.6, Command Stack=None, Command
> Version=4.2.0.0, Upgrade Direction=upgrade -> 4.2.0.0
> 2017-11-29 07:01:50,096 - Using hadoop conf dir:
> /usr/hdp/current/hadoop-client/conf
> Traceback (most recent call last):
> File
> "/var/lib/ambari-agent/cache/stacks/BigInsights/4.2/services/HBASE/package/scripts/hbase_upgrade.py",
> line 37, in
> HbaseMasterUpgrade().execute()
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
> line 367, in execute
> method(env)
> File
> "/var/lib/ambari-agent/cache/stacks/BigInsights/4.2/services/HBASE/package/scripts/hbase_upgrade.py",
> line 28, in take_snapshot
> import params
> File
> "/var/lib/ambari-agent/cache/stacks/BigInsights/4.2/services/HBASE/package/scripts/params.py",
> line 107, in
> regionserver_xmn_percent =
> expect("/configurations/hbase-env/hbase_regionserver_xmn_ratio", float)
> #AMBARI-15614
> TypeError: 'module' object is not callable
> {code}
>
>
> Upon checking the code
> [here|https://github.com/hortonworks/ambari/blob/AMBARI-2.6.1.0/ambari-server/src/main/resources/stacks/BigInsights/4.2/services/HBASE/package/scripts/params.py#L30],
> the issue seems to be with import of 'expect' module
>
> I tried the following changes in params.py:
> {code}
> L30: from resource_management.libraries.functions import expect
> L107: regionserver_xmn_percent =
> expect.expect("/configurations/hbase-env/hbase_regionserver_xmn_ratio",
> float) #AMBARI-15614
> {code}
>
> Now the snapshot command ran fine:
> {code}
> 2017-11-29 11:14:15,472 - Stack Feature Version Info: Cluster Sta
Re: Review Request 64209: Need to address HDP-GPL repo update after user accepts license in post-install scenario
> On Nov. 30, 2017, 4:12 p.m., Nate Cole wrote: > > This is making the agent-side do the logic. Shouldn't we check these > > things and not include GPL repos in the command to the agent? > > Sid Wagle wrote: > We would still need to make sure existing hosts get the changes if user > denies license during setup and then accepts it. Present behavior is that we > do not update the repo for existing hosts. Thereby, I was ok with limiting > the changes to the agent vs server and agent. The command repository is being sent down and processed on every command. - Nate --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64209/#review192365 --- On Nov. 30, 2017, 8:28 a.m., Andrew Onischuk wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64209/ > --- > > (Updated Nov. 30, 2017, 8:28 a.m.) > > > Review request for Ambari, Attila Doroszlai, Myroslav Papirkovskyy, Nate > Cole, and Sid Wagle. > > > Bugs: AMBARI-22561 > https://issues.apache.org/jira/browse/AMBARI-22561 > > > Repository: ambari > > > Description > --- > > * User denies the GPL license agreement, UI will still issue PUT call to > create the repo > * Ambari should not write HDP-GPL repo info to the ambari-hdp-1.repo > because it will break yum in local repo world. > * Then the user accepts the license by running ambari-server setup > * The repo file does not get updated on existing hosts > * We need to push the HDP-GPL repo to the repo files on existing hosts only > when the license is accepted > > > Diffs > - > > > ambari-common/src/main/python/resource_management/libraries/functions/lzo_utils.py > 68ee607b9b > > ambari-common/src/main/python/resource_management/libraries/functions/repository_util.py > f1c8ef1b04 > > ambari-common/src/main/python/resource_management/libraries/script/script.py > ccb8a547f4 > > ambari-server/src/main/java/org/apache/ambari/server/agent/CommandRepository.java > e207ac5289 > ambari-server/src/main/resources/custom_actions/scripts/install_packages.py > b819dcad96 > > ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/repo_initialization.py > 82e57aa4f4 > ambari-server/src/test/python/stacks/utils/RMFTestCase.py 80712c5ebe > > > Diff: https://reviews.apache.org/r/64209/diff/1/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Andrew Onischuk > >
Re: Review Request 64209: Need to address HDP-GPL repo update after user accepts license in post-install scenario
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64209/#review192365 --- This is making the agent-side do the logic. Shouldn't we check these things and not include GPL repos in the command to the agent? - Nate Cole On Nov. 30, 2017, 8:28 a.m., Andrew Onischuk wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64209/ > --- > > (Updated Nov. 30, 2017, 8:28 a.m.) > > > Review request for Ambari, Attila Doroszlai, Myroslav Papirkovskyy, Nate > Cole, and Sid Wagle. > > > Bugs: AMBARI-22561 > https://issues.apache.org/jira/browse/AMBARI-22561 > > > Repository: ambari > > > Description > --- > > * User denies the GPL license agreement, UI will still issue PUT call to > create the repo > * Ambari should not write HDP-GPL repo info to the ambari-hdp-1.repo > because it will break yum in local repo world. > * Then the user accepts the license by running ambari-server setup > * The repo file does not get updated on existing hosts > * We need to push the HDP-GPL repo to the repo files on existing hosts only > when the license is accepted > > > Diffs > - > > > ambari-common/src/main/python/resource_management/libraries/functions/lzo_utils.py > 68ee607b9b > > ambari-common/src/main/python/resource_management/libraries/functions/repository_util.py > f1c8ef1b04 > > ambari-common/src/main/python/resource_management/libraries/script/script.py > ccb8a547f4 > > ambari-server/src/main/java/org/apache/ambari/server/agent/CommandRepository.java > e207ac5289 > ambari-server/src/main/resources/custom_actions/scripts/install_packages.py > b819dcad96 > > ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/repo_initialization.py > 82e57aa4f4 > ambari-server/src/test/python/stacks/utils/RMFTestCase.py 80712c5ebe > > > Diff: https://reviews.apache.org/r/64209/diff/1/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Andrew Onischuk > >
Re: Review Request 64223: Oozie Fails To Restart During Upgrade Because of Missing ExtJS Library
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64223/#review192364
---
Ship it!
Ship It!
- Nate Cole
On Nov. 30, 2017, 3:28 p.m., Jonathan Hurley wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64223/
> ---
>
> (Updated Nov. 30, 2017, 3:28 p.m.)
>
>
> Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, and Nate Cole.
>
>
> Bugs: AMBARI-22568
> https://issues.apache.org/jira/browse/AMBARI-22568
>
>
> Repository: ambari
>
>
> Description
> ---
>
> 1) Deployed cluster with Ambari version: 2.6.1.0-65 and HDP version:
> 2.6.3.0-215
> 2) Express upgrade to target version 2.6.4.0-52
>
> Oozie Server Restart fails with below error:
>
> ```
> Traceback (most recent call last):
> ...
> File
> "/var/lib/ambari-agent/cache/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server.py",
> line 123, in pre_upgrade_restart
> OozieUpgrade.prepare_libext_directory(upgrade_type=upgrade_type)
> File
> "/var/lib/ambari-agent/cache/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server_upgrade.py",
> line 113, in prepare_libext_directory
> raise Fail("Unable to find any Oozie source extension files from the
> following paths {0}".format(source_ext_zip_paths))
> resource_management.core.exceptions.Fail: Unable to find any Oozie source
> extension files from the following paths ['/usr/share/HDP-oozie/ext-2.2.zip',
> '/var/lib/oozie/ext-2.2.zip']
> ```
>
> This is because we have removed the requirement of having ExtJS in the
> cluster.
>
>
> Diffs
> -
>
>
> ambari-common/src/main/python/resource_management/libraries/functions/constants.py
> b81186144c
>
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server_upgrade.py
> eb57c22834
>
> ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_features.json
> 2109a5d5e9
> ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_server.py
> 4d94723bbf
>
>
> Diff: https://reviews.apache.org/r/64223/diff/1/
>
>
> Testing
> ---
>
> Upgraded Oozie
>
> --
> Total run:1196
> Total errors:0
> Total failures:0
> OK
> [INFO]
>
> [INFO] BUILD SUCCESS
> [INFO]
>
> [INFO] Total time: 58.251 s
> [INFO] Finished at: 2017-11-30T15:16:50-05:00
> [INFO] Final Memory: 21M/619M
> [INFO]
>
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 64205: Snapshot HBase task failed during IOP migration with TypeError
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64205/#review192320
---
Ship it!
Ship It!
- Nate Cole
On Nov. 30, 2017, 10:04 a.m., Dmitro Lisnichenko wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64205/
> ---
>
> (Updated Nov. 30, 2017, 10:04 a.m.)
>
>
> Review request for Ambari, Dmytro Grinenko, Jonathan Hurley, and Nate Cole.
>
>
> Bugs: AMBARI-22558
> https://issues.apache.org/jira/browse/AMBARI-22558
>
>
> Repository: ambari
>
>
> Description
> ---
>
> *STR*
> # Deployed cluster with Ambari version: 2.2.0 and IOP version: 4.2.0.0
> # Upgrade Ambari to Target Version: 2.5.2.0-298 | Hash:
> 2453e16418fd964042452b649153dbe45f3c6009
> # Upgrade Ambari to Target Version: 2.6.1.0-64 | Hash:
> cd4db8e9ac0ea7ce14fc1253959a121688f34952
> # Register HDP-2.6.4.0-51 and call remove iop-select
> # Install the new HDP version bits and start Express Upgrade
>
> *Result*
> Snapshot HBase task failed with below error:
> {code}
> Traceback (most recent call last):
> File
> "/var/lib/ambari-agent/cache/custom_actions/scripts/ru_execute_tasks.py",
> line 157, in
> ExecuteUpgradeTasks().execute()
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
> line 367, in execute
> method(env)
> File
> "/var/lib/ambari-agent/cache/custom_actions/scripts/ru_execute_tasks.py",
> line 153, in actionexecute
> shell.checked_call(task.command, logoutput=True, quiet=True)
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
> line 72, in inner
> result = function(command, **kwargs)
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
> line 102, in checked_call
> tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy)
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
> line 150, in _call_wrapper
> result = _call(command, **kwargs_copy)
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
> line 303, in _call
> raise ExecutionFailed(err_msg, code, out, err)
> resource_management.core.exceptions.ExecutionFailed: Execution of 'source
> /var/lib/ambari-agent/ambari-env.sh ; /usr/bin/ambari-python-wrap
> /var/lib/ambari-agent/cache/stacks/BigInsights/4.2/services/HBASE/package/scripts/hbase_upgrade.py
> take_snapshot /var/lib/ambari-agent/data/command-404.json
> /var/lib/ambari-agent/cache/custom_actions
> /var/lib/ambari-agent/data/structured-out-404.json INFO
> /var/lib/ambari-agent/tmp' returned 1. 2017-11-29 07:01:50,049 - Stack
> Feature Version Info: Cluster Stack=2.6, Command Stack=None, Command
> Version=4.2.0.0, Upgrade Direction=upgrade -> 4.2.0.0
> 2017-11-29 07:01:50,096 - Using hadoop conf dir:
> /usr/hdp/current/hadoop-client/conf
> Traceback (most recent call last):
> File
> "/var/lib/ambari-agent/cache/stacks/BigInsights/4.2/services/HBASE/package/scripts/hbase_upgrade.py",
> line 37, in
> HbaseMasterUpgrade().execute()
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
> line 367, in execute
> method(env)
> File
> "/var/lib/ambari-agent/cache/stacks/BigInsights/4.2/services/HBASE/package/scripts/hbase_upgrade.py",
> line 28, in take_snapshot
> import params
> File
> "/var/lib/ambari-agent/cache/stacks/BigInsights/4.2/services/HBASE/package/scripts/params.py",
> line 107, in
> regionserver_xmn_percent =
> expect("/configurations/hbase-env/hbase_regionserver_xmn_ratio", float)
> #AMBARI-15614
> TypeError: 'module' object is not callable
> {code}
>
>
> Upon checking the code
> [here|https://github.com/hortonworks/ambari/blob/AMBARI-2.6.1.0/ambari-server/src/main/resources/stacks/BigInsights/4.2/services/HBASE/package/scripts/params.py#L30],
> the issue seems to be with import of 'expect' module
>
> I tried the following changes in params.py:
> {code}
> L30: from resource_management.libraries.functions import expect
> L107: regionserver_xmn_percent =
> expect.expect("/configurations/hbase-env/hbase_regionserver_xmn_ratio",
> float) #AMBARI-15614
> {code}
>
> Now the snapshot command ran fine:
> {code}
> 2017-11-29 11:14:15,472 - Stack Feature Version Info: Cluster Sta
Re: Review Request 63945: AMBARI-22393: UI changes for prompting user during registration failure for new OS family
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63945/#review192291 --- This has been pushed. Please close this review and appropriate JIRA. - Nate Cole On Nov. 20, 2017, 9:25 a.m., Sonia Garudi wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63945/ > --- > > (Updated Nov. 20, 2017, 9:25 a.m.) > > > Review request for Ambari and Nate Cole. > > > Bugs: AMBARI-22393 > https://issues.apache.org/jira/browse/AMBARI-22393 > > > Repository: ambari > > > Description > --- > > The bootstrap process fails with code 44 incase the ambari repository URL is > not set for a particular os type in the ambari.properties file. Checks for > the returned status code from bootstrap.py and accordingly displays a prompt > on UI for ambari reposiory URL for the failing os types. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/bootstrap/BSHostStatus.java > 3d1b31c > > ambari-server/src/main/java/org/apache/ambari/server/bootstrap/BSHostStatusCollector.java > b72ca20 > ambari-web/app/controllers/wizard/step3_controller.js 29393e3 > ambari-web/app/messages.js 390f803 > ambari-web/app/styles/wizard.less cb9eb92 > ambari-web/app/templates/wizard/step3.hbs e00012c > ambari-web/test/controllers/wizard/step3_test.js b8bc794 > > > Diff: https://reviews.apache.org/r/63945/diff/2/ > > > Testing > --- > > Tests were successful in ambari-web and ambari-server. > > > Thanks, > > Sonia Garudi > >
Re: Review Request 63945: AMBARI-22393: UI changes for prompting user during registration failure for new OS family
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63945/#review192168 --- Ship it! Ship It! - Nate Cole On Nov. 20, 2017, 9:25 a.m., Sonia Garudi wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63945/ > --- > > (Updated Nov. 20, 2017, 9:25 a.m.) > > > Review request for Ambari and Nate Cole. > > > Bugs: AMBARI-22393 > https://issues.apache.org/jira/browse/AMBARI-22393 > > > Repository: ambari > > > Description > --- > > The bootstrap process fails with code 44 incase the ambari repository URL is > not set for a particular os type in the ambari.properties file. Checks for > the returned status code from bootstrap.py and accordingly displays a prompt > on UI for ambari reposiory URL for the failing os types. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/bootstrap/BSHostStatus.java > 3d1b31c > > ambari-server/src/main/java/org/apache/ambari/server/bootstrap/BSHostStatusCollector.java > b72ca20 > ambari-web/app/controllers/wizard/step3_controller.js 29393e3 > ambari-web/app/messages.js 390f803 > ambari-web/app/styles/wizard.less cb9eb92 > ambari-web/app/templates/wizard/step3.hbs e00012c > ambari-web/test/controllers/wizard/step3_test.js b8bc794 > > > Diff: https://reviews.apache.org/r/63945/diff/2/ > > > Testing > --- > > Tests were successful in ambari-web and ambari-server. > > > Thanks, > > Sonia Garudi > >
Re: Review Request 64173: AMBARI-22550. Service install/start failure due to wrong version directory
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64173/#review192166 --- Ship it! Ship It! - Nate Cole On Nov. 29, 2017, 3 p.m., Attila Doroszlai wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64173/ > --- > > (Updated Nov. 29, 2017, 3 p.m.) > > > Review request for Ambari, Jonathan Hurley, Nate Cole, and Robert Nettleton. > > > Bugs: AMBARI-22550 > https://issues.apache.org/jira/browse/AMBARI-22550 > > > Repository: ambari > > > Description > --- > > Only send `/commandParams/version` if it is resolved: using unresolved > version eg. `2.6.3.0` only causes problems. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ExecutionCommandWrapper.java > 212363b85f1ef4f4b3b91f0ddb2fe91cd897d5dc > > ambari-server/src/test/java/org/apache/ambari/server/actionmanager/ExecutionCommandWrapperTest.java > 7630c553c0e55ffa314f02bc9c68337d502a93be > > > Diff: https://reviews.apache.org/r/64173/diff/1/ > > > Testing > --- > > * Tested cluster creation: verified that resolved version (eg. 2.6.3.0-235) > is sent in "start" commands (and version is absent in "install" commands), > and that WebHCat server is started OK. > * Tested service auto-restart for regression per steps in > [AMBARI-22025](https://issues.apache.org/jira/browse/AMBARI-22025). > * Added unit test. `Tests run: 4931, Failures: 0, Errors: 0, Skipped: 34` > > > Thanks, > > Attila Doroszlai > >
Re: Review Request 64166: History and Hive server start failed during IOP migration with AttributeError
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64166/#review192165
---
Ship it!
Ship It!
- Nate Cole
On Nov. 29, 2017, 11:55 a.m., Jonathan Hurley wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64166/
> ---
>
> (Updated Nov. 29, 2017, 11:55 a.m.)
>
>
> Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, and Nate Cole.
>
>
> Bugs: AMBARI-22547
> https://issues.apache.org/jira/browse/AMBARI-22547
>
>
> Repository: ambari
>
>
> Description
> ---
>
> *STR*
> # Deployed cluster with Ambari version: 2.4.2.0_IBM_00 and HDP version:
> 4.2.5.0- (LZO is enabled in HDFS configs)
> # Upgrade Ambari to Target Version: 2.6.1.0-64 | Hash:
> cd4db8e9ac0ea7ce14fc1253959a121688f34952
> # Register HDP-2.6.4.0-51 version and call remove iop-slect
> # Install the bits and start EU
>
> *Result*
> History server start failed with below error:
> ```
> Traceback (most recent call last):
> File
> "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py",
> line 81, in pre_upgrade_restart
> copy_to_hdfs("tez", params.user_group, params.hdfs_user,
> skip=params.sysprep_skip_copy_tarballs_hdfs)
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/copy_tarball.py",
> line 464, in copy_to_hdfs
> source_file = prepare_function()
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/copy_tarball.py",
> line 94, in _prepare_tez_tarball
> hadoop_lib_native_lzo_dir = os.path.join(stack_root, tez_version,
> "hadoop", "lib", "native")
> File "/usr/lib64/python2.6/posixpath.py", line 65, in join
> if b.startswith('/'):
> AttributeError: 'NoneType' object has no attribute 'startswith'
> ```
>
> The error looks to be due to lzo changes, as the following code has the issue:
> ```
> # if enabled, LZO GPL libraries must be copied as well
> 91 if lzo_utils.should_install_lzo():
> 92 stack_root = Script.get_stack_root()
> 93 tez_version =
> component_version.get_component_repository_version("TEZ")
> 94 hadoop_lib_native_lzo_dir = os.path.join(stack_root, tez_version,
> "hadoop", "lib", "native")
> 95
> ```
>
> **In this system, Tez is not installed and Hive is on MapReduce. However, the
> Tez binaries are installed and should be updated in the event that Tez is
> installed after upgrade.**
>
>
> Diffs
> -
>
>
> ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py
> bd1cede834
>
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py
> 2871dc0ff3
>
>
> Diff: https://reviews.apache.org/r/64166/diff/1/
>
>
> Testing
> ---
>
> Manual install & upgrade
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Review Request 64164: LZO Upgrade pre-check is missing from BigInsights Upgrade packs
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64164/ --- Review request for Ambari, Di Li and Jonathan Hurley. Bugs: AMBARI-22546 https://issues.apache.org/jira/browse/AMBARI-22546 Repository: ambari Description --- Add LZO check for BI upgrade packs Diffs - ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/nonrolling-upgrade-to-hdp-2.6.xml ecf57f0e4f ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/nonrolling-upgrade-to-hdp-2.6.xml a169eacc61 Diff: https://reviews.apache.org/r/64164/diff/1/ Testing --- Manual testing. No new unit tests. Thanks, Nate Cole
Re: Review Request 64161: Tie MapReduce to Hive and Tez For Patch Upgrades
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64161/#review192126
---
Ship it!
Ship It!
- Nate Cole
On Nov. 29, 2017, 9:49 a.m., Jonathan Hurley wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64161/
> ---
>
> (Updated Nov. 29, 2017, 9:49 a.m.)
>
>
> Review request for Ambari, Dmytro Grinenko and Nate Cole.
>
>
> Bugs: AMBARI-22545
> https://issues.apache.org/jira/browse/AMBARI-22545
>
>
> Repository: ambari
>
>
> Description
> ---
>
> Hive is currently unable to upgrade without MapReduce since the property
> {{mapreduce.admin.user.env}} cannot properly be detected from
> {{mapred-site}}.
>
> STR:
> - Install a base cluster at 2.6.3.0 with ZK, HDFS, YARN, MapR, Tez, Hive
> - Push YARN/Tez forward to 2.6.3.1-1
> - Push Hive forward to 2.6.3.2-1
> - Push MapR forward to 2.6.3.3-1
>
>
> Diffs
> -
>
>
> ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_packages.json
> 794d2b0466
>
>
> Diff: https://reviews.apache.org/r/64161/diff/1/
>
>
> Testing
> ---
>
> Manual
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 64050: Ambari UI changes to support Isilon OneFS
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64050/#review192059 --- Ship it! Ship It! - Nate Cole On Nov. 23, 2017, 10:31 a.m., Attila Magyar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64050/ > --- > > (Updated Nov. 23, 2017, 10:31 a.m.) > > > Review request for Ambari, Alexandr Antonenko, Andrii Tkach, and Nate Cole. > > > Bugs: AMBARI-22507 > https://issues.apache.org/jira/browse/AMBARI-22507 > > > Repository: ambari > > > Description > --- > > This change separates hdfs compatible file systems from the rest of the > services on the UI, and makes sure that one DFS can be selected at a time. > > > Diffs > - > > ambari-web/app/controllers/wizard/step4_controller.js 8a62f3b > ambari-web/app/mappers/stack_service_mapper.js f49b4d0 > ambari-web/app/messages.js 9ca51de > ambari-web/app/models/stack_service.js a84816e > ambari-web/app/templates/wizard/step4.hbs 5a08250 > ambari-web/test/controllers/wizard/step4_test.js 1cd4863 > > > Diff: https://reviews.apache.org/r/64050/diff/1/ > > > Testing > --- > > - tested with isilon mpack, 2 dfs: ONEFS, HDFS > - tested without isilon, 1 dfs: HDFS > > > ambari-web tests: passed > > > Thanks, > > Attila Magyar > >
Re: Review Request 64127: Remove extjs From Oozie Installations By Default
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64127/#review192058
---
Ship it!
Ship It!
- Nate Cole
On Nov. 28, 2017, 1:54 p.m., Jonathan Hurley wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64127/
> ---
>
> (Updated Nov. 28, 2017, 1:54 p.m.)
>
>
> Review request for Ambari, Nate Cole and Sid Wagle.
>
>
> Bugs: AMBARI-22535
> https://issues.apache.org/jira/browse/AMBARI-22535
>
>
> Repository: ambari
>
>
> Description
> ---
>
> GPL code was recently removed from some repositories, such as HDP-UTILS.
> Oozie uses a GPL module called extJS which is installed by default with no
> option not to install it.
>
> {code}
> 2017-11-28 17:00:17,813 - Execution of '/usr/bin/yum -d 0 -e 0 -y install
> extjs' returned 1. Error: Nothing to do
> 2017-11-28 17:00:17,814 - Failed to install package extjs. Executing
> '/usr/bin/yum clean metadata'
> 2017-11-28 17:00:18,075 - Retrying to install package extjs after 30 seconds
> {code}
>
> Since it no longer is included in the repos, it should be removed. Oozie can
> still load and start without it, but the web client will not be available.
>
> **I could not see a way to remove a dependency we need to keep for HDP 2.5
> without re-pasting the entire osSpecifics section back into HDP 2.6 with the
> extjs stuff cut out**
>
>
> Diffs
> -
>
> ambari-server/src/main/resources/stacks/HDP/2.6/services/OOZIE/metainfo.xml
> c9dfaadf37
>
>
> Diff: https://reviews.apache.org/r/64127/diff/1/
>
>
> Testing
> ---
>
> Installed Oozie on HDP 2.6 and verified SC's passed and the UI was returning
> an HTTP 200 with a page saying the UI was disabled.
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 64089: Make Ambari LDAP configuration API consistent with existing API
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64089/#review192031
---
Ship it!
ambari-server/src/main/java/org/apache/ambari/server/api/services/RootServiceComponentConfigurationService.java
Lines 97-99 (patched)
<https://reviews.apache.org/r/64089/#comment270019>
Nit: we've recently been using various builders for these:
Sets.newHashSet(...), Map builders and the like.
- Nate Cole
On Nov. 27, 2017, 3:52 p.m., Robert Levas wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64089/
> ---
>
> (Updated Nov. 27, 2017, 3:52 p.m.)
>
>
> Review request for Ambari, Attila Magyar, Balázs Bence Sári, Eugene
> Chekanskiy, Jonathan Hurley, Jayush Luniya, Nate Cole, Robert Nettleton, and
> Swapan Shridhar.
>
>
> Bugs: AMBARI-22481
> https://issues.apache.org/jira/browse/AMBARI-22481
>
>
> Repository: ambari
>
>
> Description
> ---
>
> Make Ambari configuration API consistent with existing API.
>
> The current API entry point (as of AMBARI-21307) to set and test the LDAP
> configuration is {{/api/v1/ldapconfigs}}. This should be more inline with the
> existing entry point for Ambari server related data...
> `/api/v1/services/AMBARI/components/AMBARI_SERVER`.
>
> The new API entry point for Ambari server related configuration data should
> be `/api/v1/services/AMBARI/components/AMBARI_SERVER/configurations`. For
> the LDAP-specific configuration, the configuration category is
> *"ldap-configuration"*.
>
> See AMBARI-22481 for examples on using the API.
>
> For testing the configuration, the following directives should be used:
>
> * `op=test-connection`
> * `op=test-attributes`
> * `op=detect-attributes`
>
>
> Diffs
> -
>
>
> ambari-server/src/main/java/org/apache/ambari/server/api/handlers/CreateHandler.java
> b614c5e92f
>
> ambari-server/src/main/java/org/apache/ambari/server/api/handlers/UpdateHandler.java
> 2f1af512be
>
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/ResourceInstanceFactoryImpl.java
> fecaedc459
>
> ambari-server/src/main/java/org/apache/ambari/server/api/services/RootServiceComponentConfigurationService.java
> c7c37a6956
>
> ambari-server/src/main/java/org/apache/ambari/server/api/services/ldap/AmbariConfiguration.java
> 7bac65ec83
>
> ambari-server/src/main/java/org/apache/ambari/server/api/services/ldap/LdapConfigOperation.java
> 478d4ff188
>
> ambari-server/src/main/java/org/apache/ambari/server/api/services/ldap/LdapConfigurationRequest.java
> 0e065e5f21
>
> ambari-server/src/main/java/org/apache/ambari/server/api/services/ldap/LdapConfigurationService.java
> 22784cd599
>
> ambari-server/src/main/java/org/apache/ambari/server/api/services/ldap/LdapRequestInfo.java
> eeecfeec38
>
> ambari-server/src/main/java/org/apache/ambari/server/api/services/serializers/JsonSerializer.java
> 0604d2bdc9
>
> ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/commands/StackAdvisorCommand.java
> 1b89c4f9a7
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AmbariServerConfigurationCategory.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AmbariServerConfigurationHandler.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AmbariServerLDAPConfigurationHandler.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/OperationStatusMetaData.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RootServiceComponentConfigurationHandler.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RootServiceComponentConfigurationHandlerFactory.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RootServiceComponentConfigurationResourceProvider.java
> 78078652f0
> ambari-server/src/main/java/org/apache/ambari/server/ldap/LdapModule.java
> 089da1df7b
>
> ambari-server/src/main/java/org/apache/ambari/server/ldap/domain/AmbariLdapConfigKeys.java
> da655adb57
>
> ambari-server/src/main/java/org/apache/ambari/server/ldap/domain/AmbariLdapConfiguration.java
> 6c466bae
Re: Review Request 64089: Make Ambari LDAP configuration API consistent with existing API
> On Nov. 28, 2017, 4:45 a.m., Attila Magyar wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AmbariServerConfigurationHandler.java
> > Lines 52 (patched)
> > <https://reviews.apache.org/r/64089/diff/1/?file=1901817#file1901817line52>
> >
> > Do we need this init() method? I don't think @StaticallyInject requires
> > it.
+1 for this comment.
- Nate
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64089/#review192005
---
On Nov. 27, 2017, 3:52 p.m., Robert Levas wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64089/
> ---
>
> (Updated Nov. 27, 2017, 3:52 p.m.)
>
>
> Review request for Ambari, Attila Magyar, Balázs Bence Sári, Eugene
> Chekanskiy, Jonathan Hurley, Jayush Luniya, Nate Cole, Robert Nettleton, and
> Swapan Shridhar.
>
>
> Bugs: AMBARI-22481
> https://issues.apache.org/jira/browse/AMBARI-22481
>
>
> Repository: ambari
>
>
> Description
> ---
>
> Make Ambari configuration API consistent with existing API.
>
> The current API entry point (as of AMBARI-21307) to set and test the LDAP
> configuration is {{/api/v1/ldapconfigs}}. This should be more inline with the
> existing entry point for Ambari server related data...
> `/api/v1/services/AMBARI/components/AMBARI_SERVER`.
>
> The new API entry point for Ambari server related configuration data should
> be `/api/v1/services/AMBARI/components/AMBARI_SERVER/configurations`. For
> the LDAP-specific configuration, the configuration category is
> *"ldap-configuration"*.
>
> See AMBARI-22481 for examples on using the API.
>
> For testing the configuration, the following directives should be used:
>
> * `op=test-connection`
> * `op=test-attributes`
> * `op=detect-attributes`
>
>
> Diffs
> -
>
>
> ambari-server/src/main/java/org/apache/ambari/server/api/handlers/CreateHandler.java
> b614c5e92f
>
> ambari-server/src/main/java/org/apache/ambari/server/api/handlers/UpdateHandler.java
> 2f1af512be
>
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/ResourceInstanceFactoryImpl.java
> fecaedc459
>
> ambari-server/src/main/java/org/apache/ambari/server/api/services/RootServiceComponentConfigurationService.java
> c7c37a6956
>
> ambari-server/src/main/java/org/apache/ambari/server/api/services/ldap/AmbariConfiguration.java
> 7bac65ec83
>
> ambari-server/src/main/java/org/apache/ambari/server/api/services/ldap/LdapConfigOperation.java
> 478d4ff188
>
> ambari-server/src/main/java/org/apache/ambari/server/api/services/ldap/LdapConfigurationRequest.java
> 0e065e5f21
>
> ambari-server/src/main/java/org/apache/ambari/server/api/services/ldap/LdapConfigurationService.java
> 22784cd599
>
> ambari-server/src/main/java/org/apache/ambari/server/api/services/ldap/LdapRequestInfo.java
> eeecfeec38
>
> ambari-server/src/main/java/org/apache/ambari/server/api/services/serializers/JsonSerializer.java
> 0604d2bdc9
>
> ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/commands/StackAdvisorCommand.java
> 1b89c4f9a7
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AmbariServerConfigurationCategory.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AmbariServerConfigurationHandler.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AmbariServerLDAPConfigurationHandler.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/OperationStatusMetaData.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RootServiceComponentConfigurationHandler.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RootServiceComponentConfigurationHandlerFactory.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RootServiceComponentConfigurationResourceProvider.java
> 78078652f0
> ambari-server/src/main/java/org/apache/ambari/server/ldap/LdapModule.java
> 089da1df7b
>
> ambari-server/src/main/java/org/apache/ambari/server/ldap/domain/AmbariLdapConfigKeys.java
> da655adb57
>
> ambari-se
Re: Review Request 64050: Ambari UI changes to support Isilon OneFS
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64050/#review191896 --- ambari-web/app/mappers/stack_service_mapper.js Lines 122-124 (patched) <https://reviews.apache.org/r/64050/#comment269833> HDFS should also have its type as HCFS, no? meaning, can we only EVER check for service_type === HCFS? - Nate Cole On Nov. 23, 2017, 10:31 a.m., Attila Magyar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64050/ > --- > > (Updated Nov. 23, 2017, 10:31 a.m.) > > > Review request for Ambari, Alexandr Antonenko, Andrii Tkach, and Nate Cole. > > > Bugs: AMBARI-22507 > https://issues.apache.org/jira/browse/AMBARI-22507 > > > Repository: ambari > > > Description > --- > > This change separates hdfs compatible file systems from the rest of the > services on the UI, and makes sure that one DFS can be selected at a time. > > > Diffs > - > > ambari-web/app/controllers/wizard/step4_controller.js 8a62f3b > ambari-web/app/mappers/stack_service_mapper.js f49b4d0 > ambari-web/app/messages.js 9ca51de > ambari-web/app/models/stack_service.js a84816e > ambari-web/app/templates/wizard/step4.hbs 5a08250 > ambari-web/test/controllers/wizard/step4_test.js 1cd4863 > > > Diff: https://reviews.apache.org/r/64050/diff/1/ > > > Testing > --- > > - tested with isilon mpack, 2 dfs: ONEFS, HDFS > - tested without isilon, 1 dfs: HDFS > > > ambari-web tests: passed > > > Thanks, > > Attila Magyar > >
Re: Review Request 64043: SNMP Trap should contain sysuptime field filled
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64043/#review191895 --- Ship it! Ship It! - Nate Cole On Nov. 22, 2017, 7:55 p.m., Amarnath reddy pappu wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64043/ > --- > > (Updated Nov. 22, 2017, 7:55 p.m.) > > > Review request for Ambari, Jonathan Hurley and Nate Cole. > > > Bugs: AMBARI-22458 > https://issues.apache.org/jira/browse/AMBARI-22458 > > > Repository: ambari > > > Description > --- > > SNMP Trap should contain sysuptime field filled: > > Calculating the Ambari server up time and setting it in TRAP. > > Fix: calculating the uptime using below call. > > RuntimeMXBean runtimeMXBean = ManagementFactory.getRuntimeMXBean(); > long uptimeInMillis = runtimeMXBean.getUptime(); > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/notifications/dispatchers/AmbariSNMPDispatcher.java > 1eb9977 > > > Diff: https://reviews.apache.org/r/64043/diff/1/ > > > Testing > --- > > I have done basic testing locally by printing the uptimeInMillis in the logs > - I have provided this patch to one of the customer and they confirmed that > this patch works fine and could see sysuptime in target SNMP Target server. > > > Thanks, > > Amarnath reddy pappu > >
Re: Review Request 63957: Stack advisor should disallow lzo enable without accepting license agreement.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63957/#review191756 --- Ship it! Ship It! - Nate Cole On Nov. 22, 2017, 1:09 p.m., Myroslav Papirkovskyy wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63957/ > --- > > (Updated Nov. 22, 2017, 1:09 p.m.) > > > Review request for Ambari, Andrew Onischuk, Jonathan Hurley, Nate Cole, and > Sid Wagle. > > > Bugs: AMBARI-22484 > https://issues.apache.org/jira/browse/AMBARI-22484 > > > Repository: ambari > > > Description > --- > > If the user adds com.hadoop.compression.lzo.LzoCodec to core-site property > io.compression.codec.lzo.class and the ambari.properties does not have the > allow GPL software install flag, throw validation error > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/StackAdvisorRequest.java > 64180e5039bdb74d3aaf25e140565c82a95a26c1 > > ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/commands/StackAdvisorCommand.java > 5440462b86cdcfafb1e9ab239dacfddee0b31b93 > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java > b2a68d66d777f107fdc4f5576a691e8d25ec2c92 > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackAdvisorResourceProvider.java > dd32c6badb18c1a8c02b7093813cd92ccd2fc74c > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py > c36a41f49b224a904368c7474f0f769348e65ae6 > ambari-server/src/main/resources/stacks/stack_advisor.py > 8ccbaba9db083068268951ae4e07ce08d32edcc4 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ValidationResourceProviderTest.java > cdbf2d782f86b2a30f54619c4e2a9165a020275c > ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py > 9e0070458987ee490f3918fead5ef11d201c8612 > > > Diff: https://reviews.apache.org/r/63957/diff/2/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Myroslav Papirkovskyy > >
Re: Review Request 64030: Modify AMBARI-22387 to Check for LZO + No Opt-in
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64030/#review191741 --- Ship it! Ship It! - Nate Cole On Nov. 22, 2017, 9 a.m., Dmitro Lisnichenko wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64030/ > --- > > (Updated Nov. 22, 2017, 9 a.m.) > > > Review request for Ambari, Jonathan Hurley and Nate Cole. > > > Bugs: AMBARI-22500 > https://issues.apache.org/jira/browse/AMBARI-22500 > > > Repository: ambari > > > Description > --- > > The work for AMBARI-22387 was initially created to warn the user before > performing an upgrade that LZO was enabled and would need to be managed on > their own. However, with AMBARI-22457, we now support LZO when a user opts-in > to installed GPL-licensed code. > > The warning in AMBARI-22387 should be changed to check for LZO being enabled > but the user _NOT_ opt-ing in > > > Diffs > - > > ambari-server/src/main/java/org/apache/ambari/server/checks/LZOCheck.java > 9c0286b > > ambari-server/src/test/java/org/apache/ambari/server/checks/LZOCheckTest.java > e50e936 > > > Diff: https://reviews.apache.org/r/64030/diff/1/ > > > Testing > --- > > mvn clean test > and check on live cluster > > > Thanks, > > Dmitro Lisnichenko > >
Re: Review Request 63961: Ambari upgrade failed
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63961/#review191650
---
Ship it!
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog260.java
Line 365 (original), 368 (patched)
<https://reviews.apache.org/r/63961/#comment269497>
If things are working, then I'm ok with this.
- Nate Cole
On Nov. 21, 2017, 12:56 p.m., Dmitro Lisnichenko wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63961/
> ---
>
> (Updated Nov. 21, 2017, 12:56 p.m.)
>
>
> Review request for Ambari, Jonathan Hurley and Nate Cole.
>
>
> Bugs: AMBARI-22469
> https://issues.apache.org/jira/browse/AMBARI-22469
>
>
> Repository: ambari
>
>
> Description
> ---
>
> Ambari upgrade would fail for all Ambari view servers.
> Steps to reproduce:
> 1. Install Ambari 2.5.2 and setup it as view server. (it you don't set up it
> up as view server also it fails)
> 2. now install 2.6.0
> 3. run ambari-server upgrade
> it fails out with below exception.
>
> {noformat}
> ERROR: Error executing schema upgrade, please check the server logs.
> ERROR: Error output from schema upgrade command:
> ERROR: Exception in thread "main" org.apache.ambari.server.AmbariException:
> Unable to find any CURRENT repositories.
> at
> org.apache.ambari.server.upgrade.SchemaUpgradeHelper.executeUpgrade(SchemaUpgradeHelper.java:203)
> at
> org.apache.ambari.server.upgrade.SchemaUpgradeHelper.main(SchemaUpgradeHelper.java:418)
> Caused by: org.apache.ambari.server.AmbariException: Unable to find any
> CURRENT repositories.
> at
> org.apache.ambari.server.upgrade.UpgradeCatalog260.getCurrentVersionID(UpgradeCatalog260.java:510)
> at
> org.apache.ambari.server.upgrade.UpgradeCatalog260.executeDDLUpdates(UpgradeCatalog260.java:194)
> at
> org.apache.ambari.server.upgrade.AbstractUpgradeCatalog.upgradeSchema(AbstractUpgradeCatalog.java:923)
> at
> org.apache.ambari.server.upgrade.SchemaUpgradeHelper.executeUpgrade(SchemaUpgradeHelper.java:200)
> ... 1 more
>
>
> ERROR: Ambari server upgrade failed. Please look at
> /var/log/ambari-server/ambari-server.log, for more details.
> ERROR: Exiting with exit code 11.
> REASON: Schema upgrade failed.
> {noformat}
>
> For some reason we are checking cluster_version table entries and throwing up
> above exception.
>
>
> {noformat}
> In UpgradeCatalog260.java
> public int getCurrentVersionID() throws AmbariException, SQLException {
> List currentVersionList =
> dbAccessor.getIntColumnValues(CLUSTER_VERSION_TABLE, REPO_VERSION_ID_COLUMN,
> new String[]{STATE_COLUMN}, new String[]{CURRENT}, false);
> if (currentVersionList.isEmpty()) {
> throw new AmbariException("Unable to find any CURRENT repositories.");
> } else if (currentVersionList.size() != 1) {
> throw new AmbariException("The following repositories were found to be
> CURRENT: ".concat(StringUtils.join(currentVersionList, ",")));
> }
> return currentVersionList.get(0);
> }
> {noformat}
>
>
> Diffs
> -
>
>
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog260.java
> 96ce807d5f
>
>
> Diff: https://reviews.apache.org/r/63961/diff/2/
>
>
> Testing
> ---
>
> check on live cluster
> tests passed
>
>
> Thanks,
>
> Dmitro Lisnichenko
>
>
Re: Review Request 63982: Port AMBARI-22467 to BigInsight
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63982/#review191646 --- Ship it! Ship It! - Nate Cole On Nov. 21, 2017, 5:58 a.m., Andrew Onischuk wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63982/ > --- > > (Updated Nov. 21, 2017, 5:58 a.m.) > > > Review request for Ambari, Jonathan Hurley, Myroslav Papirkovskyy, Nate Cole, > and Sid Wagle. > > > Bugs: AMBARI-22489 > https://issues.apache.org/jira/browse/AMBARI-22489 > > > Repository: ambari > > > Description > --- > > . > > > Diffs > - > > > ambari-server/src/main/resources/stacks/BigInsights/4.0/services/HDFS/metainfo.xml > 918cdb32fa > > ambari-server/src/main/resources/stacks/BigInsights/4.2/services/HDFS/metainfo.xml > 72ce2be363 > > ambari-server/src/main/resources/stacks/BigInsights/4.2/services/HDFS/package/scripts/hdfs.py > 2b158bd128 > > ambari-server/src/main/resources/stacks/BigInsights/4.2/services/HIVE/package/scripts/hive.py > 0446e1cca2 > > ambari-server/src/main/resources/stacks/BigInsights/4.2/services/OOZIE/package/scripts/oozie.py > 64a83c0ba9 > > ambari-server/src/main/resources/stacks/BigInsights/4.2/services/OOZIE/package/scripts/params.py > d1a4a66f40 > > ambari-server/src/main/resources/stacks/BigInsights/4.2/services/YARN/package/scripts/yarn.py > a284183c53 > > > Diff: https://reviews.apache.org/r/63982/diff/1/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Andrew Onischuk > >
Re: Review Request 63962: Conditionally Rebuild MapReduce and Tez Tarballs with LZO if Enabled
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63962/#review191645 --- Ship it! Ship It! - Nate Cole On Nov. 20, 2017, 2:27 p.m., Jonathan Hurley wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63962/ > --- > > (Updated Nov. 20, 2017, 2:27 p.m.) > > > Review request for Ambari, Dmytro Grinenko and Nate Cole. > > > Bugs: AMBARI-22486 > https://issues.apache.org/jira/browse/AMBARI-22486 > > > Repository: ambari > > > Description > --- > > If LZO is enabled and has been opted-in, then the Tez and MapReduce tarballs > should have LZO added to them. > > > Diffs > - > > > ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py > b05c97cdfe > > ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/tez.py > dfa650110d > > ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/configuration-mapred/mapred-site.xml > 3438c45527 > > ambari-server/src/main/resources/stacks/HDP/2.2/services/TEZ/configuration/tez-site.xml > 4ffb7a49a4 > > ambari-server/src/main/resources/stacks/HDP/2.2/services/YARN/configuration-mapred/mapred-site.xml > 084e91254c > ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/config-upgrade.xml > 45380728e3 > > > Diff: https://reviews.apache.org/r/63962/diff/1/ > > > Testing > --- > > Manual testing. > > > Thanks, > > Jonathan Hurley > >
Re: Review Request 63957: Stack advisor should disallow lzo enable without accepting license agreement.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63957/#review191532 --- Ship it! ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py Lines 531 (patched) <https://reviews.apache.org/r/63957/#comment269375> nit: "Please refer to the documentation..." - Nate Cole On Nov. 20, 2017, 12:06 p.m., Myroslav Papirkovskyy wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63957/ > --- > > (Updated Nov. 20, 2017, 12:06 p.m.) > > > Review request for Ambari, Andrew Onischuk, Jonathan Hurley, Nate Cole, and > Sid Wagle. > > > Bugs: AMBARI-22484 > https://issues.apache.org/jira/browse/AMBARI-22484 > > > Repository: ambari > > > Description > --- > > If the user adds com.hadoop.compression.lzo.LzoCodec to core-site property > io.compression.codec.lzo.class and the ambari.properties does not have the > allow GPL software install flag, throw validation error > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/StackAdvisorRequest.java > 64180e5039bdb74d3aaf25e140565c82a95a26c1 > > ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/commands/StackAdvisorCommand.java > 5440462b86cdcfafb1e9ab239dacfddee0b31b93 > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java > b2a68d66d777f107fdc4f5576a691e8d25ec2c92 > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackAdvisorResourceProvider.java > dd32c6badb18c1a8c02b7093813cd92ccd2fc74c > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py > c36a41f49b224a904368c7474f0f769348e65ae6 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ValidationResourceProviderTest.java > cdbf2d782f86b2a30f54619c4e2a9165a020275c > ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py > 9e0070458987ee490f3918fead5ef11d201c8612 > > > Diff: https://reviews.apache.org/r/63957/diff/1/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Myroslav Papirkovskyy > >
Re: Review Request 63955: Validate blueprint does not allow lzo enable without setup with license agreement.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63955/#review191531 --- Ship it! Ship It! - Nate Cole On Nov. 20, 2017, 12:06 p.m., Myroslav Papirkovskyy wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63955/ > --- > > (Updated Nov. 20, 2017, 12:06 p.m.) > > > Review request for Ambari, Andrew Onischuk, Jonathan Hurley, Nate Cole, > Robert Nettleton, and Sid Wagle. > > > Bugs: AMBARI-22480 > https://issues.apache.org/jira/browse/AMBARI-22480 > > > Repository: ambari > > > Description > --- > > Blueprint validation should fail if LZO is enabled and GPL license is not > accepted on server setup. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintResourceProvider.java > 1c127c080cd33314e47bcfff63e0c1e6d58937bc > > ambari-server/src/main/java/org/apache/ambari/server/topology/Blueprint.java > c9dc7281a092ff9f22551c9f8fc64ebca15fda36 > > ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintImpl.java > 1d917e0940b1ba5988a740005017c9d228e46f96 > > ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintValidator.java > 206d161fc694f67ae982ef668c5057b4bacec59b > > ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintValidatorImpl.java > 1aaa5ce66d1a5385018e4f29c92441e157c2be02 > > ambari-server/src/main/java/org/apache/ambari/server/topology/GPLLicenseNotAcceptedException.java > PRE-CREATION > > ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintImplTest.java > fdbc0807788377f689e19200be9d6cdc2c992690 > > ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintValidatorImplTest.java > ea6bee314192f0665d4df652cf0c401e0218026f > > > Diff: https://reviews.apache.org/r/63955/diff/1/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Myroslav Papirkovskyy > >
Re: Review Request 63916: YARN, MapReduce2, Hive, and Oozie Should Conditionally Install LZO
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63916/#review191346
---
Please also verify if BigInsight stacks are affected by this change. If that
is the case, then a separate JIRA should be opened to address that.
- Nate Cole
On Nov. 17, 2017, 9:40 a.m., Andrew Onischuk wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63916/
> ---
>
> (Updated Nov. 17, 2017, 9:40 a.m.)
>
>
> Review request for Ambari, Attila Doroszlai, Jonathan Hurley, Myroslav
> Papirkovskyy, Nate Cole, and Sid Wagle.
>
>
> Bugs: AMBARI-22467
> https://issues.apache.org/jira/browse/AMBARI-22467
>
>
> Repository: ambari
>
>
> Description
> ---
>
> Changes Summary:
> - Do not install lzo if GPL license if not accepted, instead give out
> logger.warning and skip
> - Remove installation of lzo from metainfo.xml. Since we need to install lzo
> on restart if lzo is enabled/license accepted, we still need to do this in
> Python code. No need to duplicate.
> Clients will also use that code during install.
> - Move function get_lzo_packages to lzo_utils.py
> - Add code for Yarn, MR2, Hive to install lzo, edit existing code to install
> lzo for HDFS,Oozie to use lzo_utils.py
>
> Notes:
> - Impl is Patch Upgrade friendly as uses the same code to detect
> {stack_version} as all the other packages. That code (not implemented here)
> gets version depending component specific repo info not stack specific.
>
>
> Diffs
> -
>
>
> ambari-common/src/main/python/resource_management/libraries/functions/__init__.py
> f144b2da32
>
> ambari-common/src/main/python/resource_management/libraries/functions/get_lzo_packages.py
> cfbb7d81e8
>
> ambari-common/src/main/python/resource_management/libraries/functions/package_conditions.py
> 31e78b9e60
>
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml
> 2df1b9f647
>
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs.py
> 6f702d3616
>
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/install_params.py
> fe488c3004
>
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py
> 5ca012b796
>
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive.py
> abbe59ed9f
>
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie.py
> 7d22603184
>
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/params_linux.py
> cd1f7ad36f
>
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py
> f6e8391a54
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/metainfo.xml
> 07a9ef8e94
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/metainfo.xml
> 63e85e363f
> ambari-server/src/test/python/stacks/2.2/configs/oozie-upgrade.json
> 86ca03a4a3
>
>
> Diff: https://reviews.apache.org/r/63916/diff/1/
>
>
> Testing
> ---
>
> mvn clean test
>
>
> Thanks,
>
> Andrew Onischuk
>
>
Re: Review Request 63916: YARN, MapReduce2, Hive, and Oozie Should Conditionally Install LZO
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63916/#review191345
---
We should add python tests for enabled and disabled LZO for coverage. If not
here, then please make sure there's a JIRA for it.
ambari-common/src/main/python/resource_management/libraries/functions/get_lzo_packages.py
Lines 67-68 (patched)
<https://reviews.apache.org/r/63916/#comment269131>
"Cannot install LZO. The GPL license must be explicitly enabled using
'ambari-server setup' on the Ambari host, then restart the server and try
again."
ambari-common/src/main/python/resource_management/libraries/functions/get_lzo_packages.py
Lines 77-79 (patched)
<https://reviews.apache.org/r/63916/#comment269132>
Is a zero-length possible? Should log it if that's the case.
ambari-common/src/main/python/resource_management/libraries/functions/get_lzo_packages.py
Lines 82-83 (patched)
<https://reviews.apache.org/r/63916/#comment269133>
Any key checks we need here? Use default(...)?
ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie.py
Lines 308-311 (original), 309-310 (patched)
<https://reviews.apache.org/r/63916/#comment269134>
This will happen only for Oozie servers, not the client
- Nate Cole
On Nov. 17, 2017, 9:40 a.m., Andrew Onischuk wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63916/
> ---
>
> (Updated Nov. 17, 2017, 9:40 a.m.)
>
>
> Review request for Ambari, Attila Doroszlai, Jonathan Hurley, Myroslav
> Papirkovskyy, Nate Cole, and Sid Wagle.
>
>
> Bugs: AMBARI-22467
> https://issues.apache.org/jira/browse/AMBARI-22467
>
>
> Repository: ambari
>
>
> Description
> ---
>
> Changes Summary:
> - Do not install lzo if GPL license if not accepted, instead give out
> logger.warning and skip
> - Remove installation of lzo from metainfo.xml. Since we need to install lzo
> on restart if lzo is enabled/license accepted, we still need to do this in
> Python code. No need to duplicate.
> Clients will also use that code during install.
> - Move function get_lzo_packages to lzo_utils.py
> - Add code for Yarn, MR2, Hive to install lzo, edit existing code to install
> lzo for HDFS,Oozie to use lzo_utils.py
>
> Notes:
> - Impl is Patch Upgrade friendly as uses the same code to detect
> {stack_version} as all the other packages. That code (not implemented here)
> gets version depending component specific repo info not stack specific.
>
>
> Diffs
> -
>
>
> ambari-common/src/main/python/resource_management/libraries/functions/__init__.py
> f144b2da32
>
> ambari-common/src/main/python/resource_management/libraries/functions/get_lzo_packages.py
> cfbb7d81e8
>
> ambari-common/src/main/python/resource_management/libraries/functions/package_conditions.py
> 31e78b9e60
>
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml
> 2df1b9f647
>
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs.py
> 6f702d3616
>
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/install_params.py
> fe488c3004
>
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py
> 5ca012b796
>
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive.py
> abbe59ed9f
>
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie.py
> 7d22603184
>
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/params_linux.py
> cd1f7ad36f
>
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py
> f6e8391a54
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/metainfo.xml
> 07a9ef8e94
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/metainfo.xml
> 63e85e363f
> ambari-server/src/test/python/stacks/2.2/configs/oozie-upgrade.json
> 86ca03a4a3
>
>
> Diff: https://reviews.apache.org/r/63916/diff/1/
>
>
> Testing
> ---
>
> mvn clean test
>
>
> Thanks,
>
> Andrew Onischuk
>
>
Re: Review Request 63440: Metrics for OneFS mpack
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63440/#review191246
---
Ship it!
Ship It!
- Nate Cole
On Nov. 6, 2017, 4:48 a.m., Attila Magyar wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63440/
> ---
>
> (Updated Nov. 6, 2017, 4:48 a.m.)
>
>
> Review request for Ambari, Nate Cole, Sebastian Toader, and Sid Wagle.
>
>
> Bugs: AMBARI-22339
> https://issues.apache.org/jira/browse/AMBARI-22339
>
>
> Repository: ambari
>
>
> Description
> ---
>
> OneFS puts its own hostname into the metrics data that is sent to AMS. That
> hostname is not registered to Ambari therefore AMS host metrics don't show up.
> I added an option to override the default hostname in the metrics.json file.
>
>
> For example when a metrics.json has this:
>
> "properties" : {
> "overridden_host" : "${hdfs-site/dfs.namenode.http-address}"
> },
>
> Ambari will use this hostname (instead of the component hostname) to assemble
> the URL that is used to get the metrics data from AMS.
>
> for example:
> http://c6403.ambari.apache.org:6188/ws/v1/timeline/metrics?metricNames=cpu_wio=<>
>
>
> Diffs
> -
>
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/OverriddenMetricsHostProvider.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackDefinedPropertyProvider.java
> 8c2198b
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/MetricHostProvider.java
> 28e5381
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/AMSComponentPropertyProvider.java
> 487afde
>
> ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java
> edeb4b7
>
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/MetricDefinition.java
> 635ae35
>
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/OverriddenMetricsHostProviderTest.java
> PRE-CREATION
>
> contrib/management-packs/isilon-onefs-mpack/src/main/resources/addon-services/ONEFS/1.0.0/metrics.json
> 335bfba
>
> contrib/management-packs/isilon-onefs-mpack/src/main/resources/addon-services/ONEFS/1.0.0/widgets.json
> dbde93b
>
>
> Diff: https://reviews.apache.org/r/63440/diff/2/
>
>
> Testing
> ---
>
> - checked ams host and non host metrics through the ui
>
>
> existing tests: passed
>
>
> Thanks,
>
> Attila Magyar
>
>
Re: Review Request 63876: ambari-server upgrade to 2.6.1 should surface the GPL agreement
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63876/#review191221 --- Fix it, then Ship it! Just minor typos. ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog261.java Lines 41 (patched) <https://reviews.apache.org/r/63876/#comment268886> LZO_ENABLED_JSON_KEY ambari-server/src/main/python/ambari_server/serverUpgrade.py Lines 77 (patched) <https://reviews.apache.org/r/63876/#comment268884> By saying no, ... - Nate Cole On Nov. 16, 2017, 8:27 a.m., Andrew Onischuk wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63876/ > --- > > (Updated Nov. 16, 2017, 8:27 a.m.) > > > Review request for Ambari, Attila Doroszlai, Myroslav Papirkovskyy, and Sid > Wagle. > > > Bugs: AMBARI-22454 > https://issues.apache.org/jira/browse/AMBARI-22454 > > > Repository: ambari > > > Description > --- > > * ambari-server upgrade should determine whether lzo is enabled. > * If yes: Surface the same agreement as the setup step > * If no: Provide link to the documentation to disable lzo configuration > and cleanup local repositories. > > Since this involves reading from the database we should think of sending back > data on the pipe as stdout from the upgrade jvm and the python process. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java > dac871ea28 > > ambari-server/src/main/java/org/apache/ambari/server/upgrade/SchemaUpgradeHelper.java > a2dea40f58 > > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog.java > 21273fd49a > > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog261.java > PRE-CREATION > ambari-server/src/main/python/ambari_server/serverConfiguration.py > 262e9447e5 > ambari-server/src/main/python/ambari_server/serverSetup.py 8a14066a66 > ambari-server/src/main/python/ambari_server/serverUpgrade.py 421adfc4a1 > > > Diff: https://reviews.apache.org/r/63876/diff/1/ > > > Testing > --- > > mvn clean test > > > Thanks, > > Andrew Onischuk > >
Review Request 63880: VDF defined HDP-GPL repo should be tagged appropriately
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63880/ --- Review request for Ambari, Andrew Onischuk, Dmitro Lisnichenko, and Jonathan Hurley. Bugs: AMBARI-22461 https://issues.apache.org/jira/browse/AMBARI-22461 Repository: ambari Description --- This adds tags to a repository for use by Ambari Diffs - ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java a6315f8350 ambari-server/src/main/java/org/apache/ambari/server/controller/RepositoryResponse.java 5ec46f044f ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RepositoryResourceProvider.java 3c10e43343 ambari-server/src/main/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProvider.java 3ca81d7648 ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RepositoryEntity.java 034f3bce6e ambari-server/src/main/java/org/apache/ambari/server/stack/RepoUtil.java 81b4da949f ambari-server/src/main/java/org/apache/ambari/server/state/RepositoryInfo.java bd53c11bd7 ambari-server/src/main/java/org/apache/ambari/server/state/stack/RepoTag.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/state/stack/RepositoryXml.java 79ace89380 ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelper.java a77caab32d ambari-server/src/main/resources/version_definition.xsd eaed31d582 ambari-server/src/test/java/org/apache/ambari/server/controller/internal/RepositoryResourceProviderTest.java c46cce708e ambari-server/src/test/java/org/apache/ambari/server/state/repository/VersionDefinitionTest.java 6f33c7e8c1 ambari-server/src/test/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelperTest.java 70ff801eb3 ambari-server/src/test/resources/version_definition_with_tags.xml PRE-CREATION contrib/version-builder/version_builder.py 5bf54303ed Diff: https://reviews.apache.org/r/63880/diff/1/ Testing --- Manual. Unit tests pending. Thanks, Nate Cole
Re: Review Request 63805: Make Ambari configuration API consistent with existing API.
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63805/#review191028
---
Ship it!
Ship It!
- Nate Cole
On Nov. 14, 2017, 2:47 p.m., Robert Levas wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63805/
> ---
>
> (Updated Nov. 14, 2017, 2:47 p.m.)
>
>
> Review request for Ambari, Attila Magyar, Balázs Bence Sári, Eugene
> Chekanskiy, Jonathan Hurley, Nate Cole, and Robert Nettleton.
>
>
> Bugs: AMBARI-22418
> https://issues.apache.org/jira/browse/AMBARI-22418
>
>
> Repository: ambari
>
>
> Description
> ---
>
> Make Ambari configuration API consistent with existing API.
>
> The current API entry point (as of AMBARI-21307) is
> {{/api/v1/ambariconfigs}}. This should be more inline with the existing entry
> point for Ambari server related data...
> {{/api/v1/services/AMBARI/components/AMBARI_SERVER}}.
>
> The new API entry point for Ambari server related configuration data should
> be {{/api/v1/services/AMBARI/components/AMBARI_SERVER/configurations}}.
>
> See https://issues.apache.org/jira/browse/AMBARI-22418 for details about the
> new entryoint.
>
> *NOTE*: This patch contains a fix for a stack advisor issue that was
> introduce in the patch for https://issues.apache.org/jira/browse/AMBARI-22347.
>
> *NOTE*: Many changes are due to moving `RootServiceResponseFactory.Services`
> from an inner class to `RootService` and
> `RootServiceResponseFactory.Components` from an inner class to
> `RootComponent`.
>
>
> Diffs
> -
>
>
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/ResourceInstanceFactoryImpl.java
> f5fb6e9900
>
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/RootServiceComponentResourceDefinition.java
> e8cb570c56
>
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariConfigurationRequestSwagger.java
> 2dca9f55f4
>
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariConfigurationResponseSwagger.java
> c55ac1dd60
>
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariConfigurationService.java
> 86ed666f41
>
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java
> 46ee65abea
>
> ambari-server/src/main/java/org/apache/ambari/server/api/services/RootServiceComponentConfigurationService.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/api/services/RootServiceService.java
> 5afb7dc90c
>
> ambari-server/src/main/java/org/apache/ambari/server/api/services/ldap/AmbariConfiguration.java
> e9f0b1e1a4
>
> ambari-server/src/main/java/org/apache/ambari/server/api/services/ldap/LdapConfigurationRequest.java
> 2e478c4329
>
> ambari-server/src/main/java/org/apache/ambari/server/api/services/ldap/LdapConfigurationService.java
> 00c13f6cfa
>
> ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/commands/StackAdvisorCommand.java
> 91edf867ae
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java
> 1425e1bd32
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/MaintenanceStateHelper.java
> d9a585f630
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/ResourceProviderFactory.java
> 711ae10f7e
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/RootComponent.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/RootService.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/RootServiceResponseFactory.java
> ad9ed202dc
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractControllerResourceProvider.java
> 1dc0841d19
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AmbariConfigurationResourceProvider.java
> a24400dffd
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RootServiceComponentConfigurationResourceProvider.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RootServiceComponentPropertyProvider.java
> 433c1fae90
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RootServiceComponentRe
Review Request 63822: Oozie Jobs on Hive Fail With Missing Tarball
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63822/
---
Review request for Ambari and Jonathan Hurley.
Repository: ambari
Description
---
STR:
- Install an HDP 2.6.3.0 cluster with Oozie, HIve, MapR, Yarn, ZK, HDFS
- Patch only Oozie to a new version, such as 2.6.3.1-1
- Attempt to run an Oozie job on Hive
The workflow job will fail with:
{code}
org.apache.oozie.action.ActionExecutorException: JA008: File does not exist:
hdfs://jhurley-pu-1.openstacklocal:8020/hdp/apps/2.6.3.1-1/mapreduce/mapreduce.tar.gz
at
org.apache.oozie.action.ActionExecutor.convertExceptionHelper(ActionExecutor.java:457)
at
org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:441)
at
org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1211)
at
org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1382)
at
org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:234)
at
org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:65)
at org.apache.oozie.command.XCommand.call(XCommand.java:287)
at
org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:331)
at
org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:260)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:178)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{code}
This is because Oozie appears to be trying to use a version of the MapReduce
tarball which matches its version. However, since it does not ship its version,
it cannot find it.
Diffs
-
ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_packages.json
68e6bf806c
Diff: https://reviews.apache.org/r/63822/diff/1/
Testing
---
Manual. Follow STR, then ran an oozie-on-hive job to confirm results.
No automated tests, as they exist already.
Thanks,
Nate Cole
Re: Review Request 63807: Add Native Libraries To Tez Tarball
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63807/#review191022
---
Ship it!
Ship It!
- Nate Cole
On Nov. 14, 2017, 4:21 p.m., Jonathan Hurley wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63807/
> ---
>
> (Updated Nov. 14, 2017, 4:21 p.m.)
>
>
> Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, and Nate Cole.
>
>
> Bugs: AMBARI-22444
> https://issues.apache.org/jira/browse/AMBARI-22444
>
>
> Repository: ambari
>
>
> Description
> ---
>
> As part of the investigation for getting patch upgrades working in Ambari
> 2.6.1, it was determined that the Tez tarball will need to have the hadoop
> native libraries added to it so that they can be detected from the tarball.
>
> STR:
> - Install ZK, MapR, Tez, Yarn, Hive
> - Enable a non-LZO codec, like Snappy
> - Patch Hive to a new version
> - Change the following properties in {{tez-site}}:
> -- tez.am.launch.env = LD_LIBRARY=./tezlib/lib/native
> -- tez.task.launch.env = LD_LIBRARY=./tezlib/lib/native
>
> When Hive commands run, they will attempt to load the native snappy libraries
> from the Tez tarball and will fail with:
> {code}
> Caused by: java.io.IOException: Unable to get CompressorType for codec
> (org.apache.hadoop.io.compress.SnappyCodec). This is most likely due to
> missing native libraries for the codec.
> at
> org.apache.tez.runtime.library.common.sort.impl.ExternalSorter.(ExternalSorter.java:217)
> {code}
>
> In order to fix this, the Tez tarball should include the native hadoop
> libraries as well:
> {code}
> ??? tez
> ? ??? lib
> ? ? ??? native
> ? ? ? ??? libhadoop.a
> ? ? ? ??? libhadoop.so -> libhadoop.so.1.0.0
> ? ? ? ??? libhadoop.so.1.0.0
> ? ? ? ??? libhadooppipes.a
> ? ? ? ??? libhadooputils.a
> ? ? ? ??? libhdfs.a
> ? ? ? ??? libhdfs.so -> libhdfs.so.0.0.0
> ? ? ? ??? libhdfs.so.0.0.0
> ? ? ? ??? libsnappy.so -> libsnappy.so.1.1.4
> ? ? ? ??? libsnappy.so.1 -> libsnappy.so.1.1.4
> ? ? ? ??? libsnappy.so.1.1.4
> {code}
>
>
> Diffs
> -
>
>
> ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py
> 03b6213ac4
>
> ambari-common/src/main/python/resource_management/libraries/functions/tar_archive.py
> c682c3e24a
>
>
> Diff: https://reviews.apache.org/r/63807/diff/1/
>
>
> Testing
> ---
>
> Manual testing to ensure the file is created and uploaded.
>
> UNIT TESTS PENDING...
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 62656: When the cluster is kerberoed, the config's group of hbase and kafka is root, Maybe should be user.group be better.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62656/#review190836 --- What is the status of this review? - Nate Cole On Sept. 28, 2017, 5:05 a.m., zhangxiaolu zhangxiaolu wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/62656/ > --- > > (Updated Sept. 28, 2017, 5:05 a.m.) > > > Review request for Ambari, Andrii Babiichuk, Alejandro Fernandez, Denys > Buzhor, Dmitro Lisnichenko, Dmytro Sen, Jaimin Jetly, Nate Cole, Richard > Zang, Srimanth Gunturi, Sid Wagle, Vitalyi Brodetskyi, and Yusaku Sako. > > > Bugs: AMBARI-22078 > https://issues.apache.org/jira/browse/AMBARI-22078 > > > Repository: ambari > > > Description > --- > > When the cluster is kerberoed, the config's group is root,Maybe should be > user.group be better. > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/hbase.py > cec6b2a > > ambari-server/src/main/resources/common-services/HBASE/2.0.0.3.0/package/scripts/hbase.py > fced4fc > > ambari-server/src/main/resources/common-services/KAFKA/0.10.0.3.0/package/scripts/kafka.py > 3f9d109 > > > Diff: https://reviews.apache.org/r/62656/diff/1/ > > > Testing > --- > > done > > > Thanks, > > zhangxiaolu zhangxiaolu > >
Re: Review Request 63749: Quicklinks for OneFS
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63749/#review190834
---
Ship it!
Ship It!
- Nate Cole
On Nov. 13, 2017, 3:56 a.m., Attila Magyar wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63749/
> ---
>
> (Updated Nov. 13, 2017, 3:56 a.m.)
>
>
> Review request for Ambari, Andrii Babiichuk, Andrii Tkach, Balázs Bence Sári,
> and Nate Cole.
>
>
> Bugs: AMBARI-22424
> https://issues.apache.org/jira/browse/AMBARI-22424
>
>
> Repository: ambari
>
>
> Description
> ---
>
> OneFS mpack should have a quicklink definition with links that point to the
> OneFS admin UI.
>
> OneFS is a client only service and the hostname of OneFS admin page is an
> external host. The quicklink engine needed to be improved to allow overriding
> hostnames.
>
> For example the following quicklink will point to http://scisilon.fqdn if
> core-site/fs.defaultFS=hdfs://scisilon.fqdn.
>
> "links": [
> {
> "component_name" : "ONEFS_CLIENT",
> "name": "onefs_web_ui",
> "label": "OneFS Web UI",
> "url": "%@://%@",
> "host": {
> "http_property": "fs.defaultFS",
> "site": "core-site"
> }
> }
> ]
> }
>
>
> Diffs
> -
>
>
> ambari-server/src/main/java/org/apache/ambari/server/state/quicklinks/Host.java
> PRE-CREATION
>
> ambari-server/src/main/java/org/apache/ambari/server/state/quicklinks/Link.java
> 1d2e712
>
> ambari-server/src/test/java/org/apache/ambari/server/stack/QuickLinksConfigurationModuleTest.java
> aac278d
> ambari-server/src/test/resources/child_quicklinks_to_override.json 4309532
> ambari-web/app/views/common/quick_view_link_view.js 5888acb
> ambari-web/test/views/common/quick_link_view_test.js 1998a9c
>
> contrib/management-packs/isilon-onefs-mpack/src/main/resources/addon-services/ONEFS/1.0.0/metainfo.xml
> 82e2022
>
> contrib/management-packs/isilon-onefs-mpack/src/main/resources/addon-services/ONEFS/1.0.0/quicklinks/quicklinks.json
> PRE-CREATION
>
>
> Diff: https://reviews.apache.org/r/63749/diff/1/
>
>
> Testing
> ---
>
> - installed mpack and clicked on quicklinks
>
> existing tests: passed
>
>
> Thanks,
>
> Attila Magyar
>
>
Re: Review Request 63707: Simplify Ambari configuration tables from AMBARI-21307
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63707/#review190832 --- Ship it! Ship It! - Nate Cole On Nov. 10, 2017, 6:24 a.m., Robert Levas wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63707/ > --- > > (Updated Nov. 10, 2017, 6:24 a.m.) > > > Review request for Ambari, Attila Magyar, Balázs Bence Sári, Eugene > Chekanskiy, Jonathan Hurley, Nate Cole, Robert Nettleton, and Swapan Shridhar. > > > Bugs: AMBARI-22347 > https://issues.apache.org/jira/browse/AMBARI-22347 > > > Repository: ambari > > > Description > --- > > Simplify Ambari configuration tables from AMBARI-21307. > > configuration_base and ambari_configuration tables were introduced in > AMBARI-21307; however, these tables are more complex than needed. Future > architecture changes will eventually negate the need to have a shard > configuration structure as proposed. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariConfigurationRequestSwagger.java > 5e8094e9c7 > > ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariConfigurationService.java > 38ae7669db > > ambari-server/src/main/java/org/apache/ambari/server/api/services/ldap/AmbariConfiguration.java > b5cc9212da > > ambari-server/src/main/java/org/apache/ambari/server/api/services/ldap/LdapConfigurationService.java > 13f8835655 > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AmbariConfigurationResourceProvider.java > 4f4cc7070d > > ambari-server/src/main/java/org/apache/ambari/server/events/AmbariEvent.java > 0f9ff52147 > > ambari-server/src/main/java/org/apache/ambari/server/events/AmbariLdapConfigChangedEvent.java > 48799d793b > > ambari-server/src/main/java/org/apache/ambari/server/ldap/domain/AmbariLdapConfiguration.java > 8b26cd3e29 > > ambari-server/src/main/java/org/apache/ambari/server/ldap/service/AmbariLdapConfigurationProvider.java > c88d420e9a > > ambari-server/src/main/java/org/apache/ambari/server/ldap/service/ads/LdapConnectionTemplateFactory.java > 8467af08b6 > > ambari-server/src/main/java/org/apache/ambari/server/orm/dao/AmbariConfigurationDAO.java > 83293efb82 > > ambari-server/src/main/java/org/apache/ambari/server/orm/entities/AmbariConfigurationEntity.java > c9f4695469 > > ambari-server/src/main/java/org/apache/ambari/server/orm/entities/AmbariConfigurationEntityPK.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/orm/entities/ConfigurationBaseEntity.java > 9ad30d7d1a > > ambari-server/src/main/java/org/apache/ambari/server/upgrade/SchemaUpgradeHelper.java > 8812ef5bfc > > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog300.java > d3e924e66c > ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql 7d634941d9 > ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql af17353ff3 > ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 89c7971303 > ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 3d2bd3a107 > ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql > 55a6c61017 > ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql e5e8af59c3 > ambari-server/src/main/resources/META-INF/persistence.xml 67eef705af > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/AmbariConfigurationResourceProviderTest.java > c2a14218e7 > > ambari-server/src/test/java/org/apache/ambari/server/orm/dao/AmbariConfigurationDAOTest.java > PRE-CREATION > > ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog300Test.java > bd8f5cbd57 > > > Diff: https://reviews.apache.org/r/63707/diff/1/ > > > Testing > --- > > Manually tested > > # Local test results: > > ``` > [INFO] > > [INFO] BUILD SUCCESS > [INFO] > > [INFO] Total time: 26:56 min > [INFO] Finished at: 2017-11-09T17:32:37-05:00 > [INFO] Final Memory: 98M/1957M > [INFO] > > ``` > > > # Jenkins test results: PENDING > > > Thanks, > > Robert Levas > >
Re: Review Request 63666: AMBARI-22327 Allow different OS types and versions for Hybrid setup
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63666/#review190831 --- This has been pushed to the feature branch. Please close the review and associated JIRA. - Nate Cole On Nov. 8, 2017, 9:27 a.m., Yussuf Shaikh wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63666/ > --- > > (Updated Nov. 8, 2017, 9:27 a.m.) > > > Review request for Ambari and Nate Cole. > > > Bugs: AMBARI-22327 > https://issues.apache.org/jira/browse/AMBARI-22327 > > > Repository: ambari > > > Description > --- > > For setting up Ambari hybrid cluster we need to allow different OS types for > agent hosts as compared to the server OS type. > The strict check during bootstrap needs to be changed in os_check_type.py > > > Diffs > - > > ambari-server/src/main/python/os_check_type.py 34de34b > ambari-server/src/test/python/TestOSCheck.py 9aba78b > > > Diff: https://reviews.apache.org/r/63666/diff/1/ > > > Testing > --- > > test_os_type_check (TestOSCheck.TestOSCheck) ... ok > > > Thanks, > > Yussuf Shaikh > >
Review Request 63713: Restart all required services failed when running cluster with several patch upgrades applied
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63713/ --- Review request for Ambari, Dmitro Lisnichenko and Jonathan Hurley. Bugs: AMBARI-22411 https://issues.apache.org/jira/browse/AMBARI-22411 Repository: ambari Description --- Small refactor to pass StackID instead of RepositoryVersionEntity, negating the check that made this call fail. Diffs - ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java d8712f2c75 ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java 5ea4bbdf3d ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 1ec790f7cc ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_packages.json 23c6d32dff ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerImplTest.java c80620f6a3 Diff: https://reviews.apache.org/r/63713/diff/1/ Testing --- Manual. Unit tests pending. Thanks, Nate Cole
Re: Review Request 63684: Patch Upgrades Broken For Clients Due To Versioned LD Library
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63684/#review190616
---
Ship it!
Do we need to consider other upgrades, e.g. from HDP-2.5 to -2.6 as well?
- Nate Cole
On Nov. 8, 2017, 4:22 p.m., Jonathan Hurley wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63684/
> ---
>
> (Updated Nov. 8, 2017, 4:22 p.m.)
>
>
> Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, and Nate Cole.
>
>
> Bugs: AMBARI-22386
> https://issues.apache.org/jira/browse/AMBARI-22386
>
>
> Repository: ambari
>
>
> Description
> ---
>
> When services are patched to different versions, jobs being run in containers
> are unable to pickup the correct JARs from the {{LD_LIBRARY_PATH}} passed in
> via the job.
>
> Consider the following situation:
> - Base cluster installed at 2.6.3.0-1
> - Enable some sort of codec, like snappy
> - YARN patched to 2.6.3.1-1
> - Hive patched to 2.6.3.2-1
>
> When Hive goes to run jobs on Tez or when jobs are run on MapR, whatever is
> sending the job is resolving the {{hdp.version}} to its own version. The box
> that the job runs on, however, might not have that specific versioned
> directory.
>
> The problem can only be solved externally by providing a way to ensure that
> the native libraries are on every machine which could potentially run jobs.
> We could install bits on every host, but that would defeat the space-saving
> purposes of patch upgrades.
>
> Another solution is to ensure that the Tez and MapR tarballs have the
> required libraries and that those libraries are added to the classpath of
> launched jobs.
>
>
> Diffs
> -
>
>
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/configuration-mapred/mapred-site.xml
> a7d8cd67bd
>
> ambari-server/src/main/resources/stacks/HDP/2.2/services/TEZ/configuration/tez-site.xml
> 1427a6faae
>
> ambari-server/src/main/resources/stacks/HDP/2.2/services/YARN/configuration-mapred/mapred-site.xml
> 4ad08cebf9
> ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/config-upgrade.xml
> 0df0334c53
>
> ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/nonrolling-upgrade-2.6.xml
> b4e3745099
> ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/upgrade-2.6.xml
> 98223904c8
>
>
> Diff: https://reviews.apache.org/r/63684/diff/2/
>
>
> Testing
> ---
>
> Manual upgrade on patched cluster.
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 63516: [Preview] Remove properties.json And Switch To Adding Properties to ResourceProviders Dynamically
> On Nov. 2, 2017, 7:42 p.m., Nate Cole wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ActionResourceProvider.java
> > Lines 69-72 (patched)
> > <https://reviews.apache.org/r/63516/diff/1/?file=1879216#file1879216line71>
> >
> > I don't think we ever change this map do we? Can use
> > ImmutableMap.builder().put()...build()
> >
> > Or, simple ones like this can be Collections.singletonMap(key, value)
>
> Dmitro Lisnichenko wrote:
> these changes would require a big patch. Should I fix all providers, or
> only ones affected by a current patch?
Just affected by current patch.
> On Nov. 2, 2017, 7:42 p.m., Nate Cole wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ActionResourceProvider.java
> > Lines 77-87 (patched)
> > <https://reviews.apache.org/r/63516/diff/1/?file=1879216#file1879216line79>
> >
> > Recommend Sets.newHashSet(String...) - it's much cleaner looking and
> > avoids an added static{} block.
>
> Dmitro Lisnichenko wrote:
> these changes would require a big patch. Should I fix all providers, or
> only ones affected by a current patch?
Just affected by current patch.
- Nate
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63516/#review189988
---
On Nov. 9, 2017, 4:58 a.m., Dmitro Lisnichenko wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63516/
> ---
>
> (Updated Nov. 9, 2017, 4:58 a.m.)
>
>
> Review request for Ambari, Jonathan Hurley and Nate Cole.
>
>
> Bugs: AMBARI-22353
> https://issues.apache.org/jira/browse/AMBARI-22353
>
>
> Repository: ambari
>
>
> Description
> ---
>
> Legacy/ancient ResourceProviders use the {{properties.json}} file to govern
> which properties can be used with the provider. This seems like excessive
> decoupling without any benefit and usually leads to runtime errors when new
> or removed properties are forgotten.
>
> This file should be removed and the providers should be registering the known
> properties on their own.
>
>
> Diffs
> -
>
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/ResourceProviderFactory.java
> 711ae10f7e
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractAuthorizedResourceProvider.java
> a24c6394ac
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractControllerResourceProvider.java
> 1dc0841d19
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractDRResourceProvider.java
> 953a2a7c77
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ActionResourceProvider.java
> 4e618e7ff6
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ActiveWidgetLayoutResourceProvider.java
> 389f0b2bf2
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AlertDefinitionResourceProvider.java
> 32f1fd8f4f
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AlertGroupResourceProvider.java
> cf4e7475b3
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AlertHistoryResourceProvider.java
> 8d716eabe0
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AlertNoticeResourceProvider.java
> 8ffd596cfa
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AlertResourceProvider.java
> c71651e5ff
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AlertTargetResourceProvider.java
> 43ee7fe7c7
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AmbariConfigurationResourceProvider.java
> 4f4cc7070d
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintResourceProvider.java
> 73d24908ea
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClientConfigResourceProvider.java
> a2a49d7cb5
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterKerberosDescriptorResourceProvider.java
> 2678d55975
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterResourceProvider.java
> 54c83600b7
>
> ambari-server/src/main/j
Re: Review Request 63695: Create a Pre-Upgrade Check Warning About LZO
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63695/#review190578
---
Should have a test.
ambari-server/src/main/java/org/apache/ambari/server/checks/CheckDescription.java
Lines 331 (patched)
<https://reviews.apache.org/r/63695/#comment268056>
Should be a cluster check - this isn't specific to any service (even though
core-site is associated with HDFS)
ambari-server/src/main/java/org/apache/ambari/server/checks/CheckDescription.java
Lines 332 (patched)
<https://reviews.apache.org/r/63695/#comment268053>
"LZO Codec Check"
ambari-server/src/main/java/org/apache/ambari/server/checks/LZOCheck.java
Lines 34-35 (patched)
<https://reviews.apache.org/r/63695/#comment268058>
Copy error :)
ambari-server/src/main/java/org/apache/ambari/server/checks/LZOCheck.java
Lines 70-72 (patched)
<https://reviews.apache.org/r/63695/#comment268055>
Elsewhere in the code "io.compression.codecs" is checked to see if it
contains "com.hadoop.compression.lzo"
We should check that one first as it's more consistent with the other
checks.
ambari-server/src/main/java/org/apache/ambari/server/checks/LZOCheck.java
Lines 76 (patched)
<https://reviews.apache.org/r/63695/#comment268057>
Not service-specific
- Nate Cole
On Nov. 9, 2017, 5:15 a.m., Dmitro Lisnichenko wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63695/
> ---
>
> (Updated Nov. 9, 2017, 5:15 a.m.)
>
>
> Review request for Ambari, Jonathan Hurley and Nate Cole.
>
>
> Bugs: AMBARI-22387
> https://issues.apache.org/jira/browse/AMBARI-22387
>
>
> Repository: ambari
>
>
> Description
> ---
>
> Ambari has removed its native support of distributing and installing LZO when
> the LZO codecs are enabled in {{core-site}}. For existing clusters where LZO
> is enabled, this means that performing an upgrade will now require manual
> user intervention to get the LZO packages installed.
>
> A pre-upgrade check should be created which checks to see if LZO is enabled
> in the cluster and then produces a {{WARNING}} to the user letting them know
> that before upgrading, they'd need to distribute the appropriate LZO packages.
>
>
> Diffs
> -
>
>
> ambari-server/src/main/java/org/apache/ambari/server/checks/CheckDescription.java
> f5e4ff9061
> ambari-server/src/main/java/org/apache/ambari/server/checks/LZOCheck.java
> PRE-CREATION
>
>
> Diff: https://reviews.apache.org/r/63695/diff/2/
>
>
> Testing
> ---
>
> checking
>
>
> Thanks,
>
> Dmitro Lisnichenko
>
>
