subject:"Review Request 51238\: Implementation for AMBARI\-15538\: Support service\-specific repo for add\-on services"

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

2016-09-13 Thread Jayush Luniya


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51238/#review148773
---


Ship it!




Verfied following scenarios
1. Add additional repo entry in the stack's repoinfo.xml. This addresses 
AMBARI-17285.
2. Verified adding a repoinfo.xml at custom-service level (service level 
extension point) adds the repo to list of repos for all stacks in the 
hierarchy. (i.e. SOLR service when added to HDP-2.4 as add-on service will also 
lead to the repo to be added to HDP-2.5)
3. Verified that if we have MYSERVICE/1.0 -> HDP-2.4 and MYSERVICE/2.0 -> 
HDP-2.5 and the 2 service versions of the service have different repoinfo.xml 
then the correct repoinfo is added to the respective stacks
4. Verified that repoinfo.xml added to a service definition in an extension is 
processed when the extension is linked and refreshed.


NOTE: When backporting this patch to 2.4/2.5 branches, we will have conflicts 
as the patch for AMBARI-18325 is in trunk, but not in those branches.

- Jayush Luniya


On Sept. 13, 2016, 9:31 a.m., Balázs Bence Sári wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51238/
> ---
> 
> (Updated Sept. 13, 2016, 9:31 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya, Nate Cole, Sumit Mohanty, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-15538
> https://issues.apache.org/jira/browse/AMBARI-15538
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Implementation contains the following things:
> - On startup, add-on service repos are loaded into the stack model
> - On startup, add-on service repos are merged into the VDF's downloaded by 
> LatestRepoCallable
> - On startup, if the is an existing cluster, it's repository version entity 
> is potentially updated with new add-on service repos.
> - Repository definitions contain two new optional fields: service_name and 
> service_version. (null for stack repositories) 
> - Small changes on the Mictrosoft-R mpack (supoorts HDP-2.4 and 2.5)
> - New unit tests
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java
>  7403dba 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  1d82928 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  bae4ab6 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProvider.java
>  02fc2ec 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/RepoUtil.java 
> PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java 
> bc94104 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java
>  3acc617 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
> 1fee589 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java
>  7bcd08b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartup.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/RepositoryInfo.java
>  83a 
>   ambari-server/src/main/java/org/apache/ambari/server/state/StackInfo.java 
> 14ff9de 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelper.java
>  d26b834 
>   ambari-server/src/main/resources/version_definition.xsd bd49028 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/RepoUtilTest.java 
> PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackManagerCommonServicesTest.java
>  1d73ff3 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackModuleTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartupTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/resources/common-services/ADDON/1.0/configuration/addon-env.xml
>  PRE-CREATION 
>   ambari-server/src/test/resources/common-services/ADDON/1.0/metainfo.xml 
> PRE-CREATION 
>   
> ambari-server/src/test/resources/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartupTest_initialRepos.json
>  PRE-CREATION 
>   
> ambari-server/src/test/resources/stacks_with_common_services/HDP/0.2/services/ADDON/metainfo.xml
>  PRE-CREATION 
>   
> ambari-server/src/test/resources/stacks_with_common_services/HDP/0.2/services/ADDON/repos/repoinfo.xml
>  PRE-CREATION 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/configuration/microsoft-r-env.xml
>  PRE-CREATION 
>   
>

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

2016-09-13 Thread Nate Cole


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51238/#review148709
---


Ship it!




Ship It!

- Nate Cole


On Sept. 13, 2016, 5:31 a.m., Balázs Bence Sári wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51238/
> ---
> 
> (Updated Sept. 13, 2016, 5:31 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya, Nate Cole, Sumit Mohanty, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-15538
> https://issues.apache.org/jira/browse/AMBARI-15538
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Implementation contains the following things:
> - On startup, add-on service repos are loaded into the stack model
> - On startup, add-on service repos are merged into the VDF's downloaded by 
> LatestRepoCallable
> - On startup, if the is an existing cluster, it's repository version entity 
> is potentially updated with new add-on service repos.
> - Repository definitions contain two new optional fields: service_name and 
> service_version. (null for stack repositories) 
> - Small changes on the Mictrosoft-R mpack (supoorts HDP-2.4 and 2.5)
> - New unit tests
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java
>  7403dba 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  1d82928 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  bae4ab6 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProvider.java
>  02fc2ec 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/RepoUtil.java 
> PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java 
> bc94104 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java
>  3acc617 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
> 1fee589 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java
>  7bcd08b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartup.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/RepositoryInfo.java
>  83a 
>   ambari-server/src/main/java/org/apache/ambari/server/state/StackInfo.java 
> 14ff9de 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelper.java
>  d26b834 
>   ambari-server/src/main/resources/version_definition.xsd bd49028 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/RepoUtilTest.java 
> PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackManagerCommonServicesTest.java
>  1d73ff3 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackModuleTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartupTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/resources/common-services/ADDON/1.0/configuration/addon-env.xml
>  PRE-CREATION 
>   ambari-server/src/test/resources/common-services/ADDON/1.0/metainfo.xml 
> PRE-CREATION 
>   
> ambari-server/src/test/resources/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartupTest_initialRepos.json
>  PRE-CREATION 
>   
> ambari-server/src/test/resources/stacks_with_common_services/HDP/0.2/services/ADDON/metainfo.xml
>  PRE-CREATION 
>   
> ambari-server/src/test/resources/stacks_with_common_services/HDP/0.2/services/ADDON/repos/repoinfo.xml
>  PRE-CREATION 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/configuration/microsoft-r-env.xml
>  PRE-CREATION 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/package/scripts/microsoft_r.py
>  61ea96b 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/custom-services/MICROSOFT_R/8.0.0/repos/repoinfo.xml
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/51238/diff/
> 
> 
> Testing
> ---
> 
> - Manually tested
> - Wrote new unit tests
> - All unit tests passed except two which were failing in CI builds as well.
> 
> 
> Thanks,
> 
> Balázs Bence Sári
> 
>

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

2016-09-13 Thread Balázs Bence Sári



> On Sept. 7, 2016, 6:35 p.m., Nate Cole wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java,
> >  lines 42-43
> > 
> >
> > Syntax.

Fixed.


> On Sept. 7, 2016, 6:35 p.m., Nate Cole wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java,
> >  lines 48-49
> > 
> >
> > Odd syntax.  Usually see:
> > 
> > @Nullable
> > private String repoDir;

Fixed.


- Balázs Bence


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51238/#review148053
---


On Sept. 13, 2016, 9:31 a.m., Balázs Bence Sári wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51238/
> ---
> 
> (Updated Sept. 13, 2016, 9:31 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya, Nate Cole, Sumit Mohanty, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-15538
> https://issues.apache.org/jira/browse/AMBARI-15538
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Implementation contains the following things:
> - On startup, add-on service repos are loaded into the stack model
> - On startup, add-on service repos are merged into the VDF's downloaded by 
> LatestRepoCallable
> - On startup, if the is an existing cluster, it's repository version entity 
> is potentially updated with new add-on service repos.
> - Repository definitions contain two new optional fields: service_name and 
> service_version. (null for stack repositories) 
> - Small changes on the Mictrosoft-R mpack (supoorts HDP-2.4 and 2.5)
> - New unit tests
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java
>  7403dba 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  1d82928 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  bae4ab6 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProvider.java
>  02fc2ec 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/RepoUtil.java 
> PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java 
> bc94104 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java
>  3acc617 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
> 1fee589 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java
>  7bcd08b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartup.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/RepositoryInfo.java
>  83a 
>   ambari-server/src/main/java/org/apache/ambari/server/state/StackInfo.java 
> 14ff9de 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelper.java
>  d26b834 
>   ambari-server/src/main/resources/version_definition.xsd bd49028 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/RepoUtilTest.java 
> PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackManagerCommonServicesTest.java
>  1d73ff3 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackModuleTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartupTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/resources/common-services/ADDON/1.0/configuration/addon-env.xml
>  PRE-CREATION 
>   ambari-server/src/test/resources/common-services/ADDON/1.0/metainfo.xml 
> PRE-CREATION 
>   
> ambari-server/src/test/resources/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartupTest_initialRepos.json
>  PRE-CREATION 
>   
> ambari-server/src/test/resources/stacks_with_common_services/HDP/0.2/services/ADDON/metainfo.xml
>  PRE-CREATION 
>   
> ambari-server/src/test/resources/stacks_with_common_services/HDP/0.2/services/ADDON/repos/repoinfo.xml
>  PRE-CREATION 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/configuration/microsoft-r-env.xml
>  PRE-CREATION 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/package/scripts/microsoft_r.py
>  61ea96b 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/custom-services/MICROSOFT_R/8.0.0/repos/repoinfo.xml
>  PRE-CREATION 
> 
> Diff:

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

2016-09-13 Thread Balázs Bence Sári


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51238/
---

(Updated Sept. 13, 2016, 9:31 a.m.)


Review request for Ambari, Jayush Luniya, Nate Cole, Sumit Mohanty, and 
Sebastian Toader.


Changes
---

Minor formatting fixes


Bugs: AMBARI-15538
https://issues.apache.org/jira/browse/AMBARI-15538


Repository: ambari


Description
---

Implementation contains the following things:
- On startup, add-on service repos are loaded into the stack model
- On startup, add-on service repos are merged into the VDF's downloaded by 
LatestRepoCallable
- On startup, if the is an existing cluster, it's repository version entity is 
potentially updated with new add-on service repos.
- Repository definitions contain two new optional fields: service_name and 
service_version. (null for stack repositories) 
- Small changes on the Mictrosoft-R mpack (supoorts HDP-2.4 and 2.5)
- New unit tests


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java
 7403dba 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
 1d82928 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
 bae4ab6 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProvider.java
 02fc2ec 
  ambari-server/src/main/java/org/apache/ambari/server/stack/RepoUtil.java 
PRE-CREATION 
  ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java 
bc94104 
  
ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java 
3acc617 
  ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
1fee589 
  
ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java
 7bcd08b 
  
ambari-server/src/main/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartup.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/state/RepositoryInfo.java 
83a 
  ambari-server/src/main/java/org/apache/ambari/server/state/StackInfo.java 
14ff9de 
  
ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelper.java
 d26b834 
  ambari-server/src/main/resources/version_definition.xsd bd49028 
  ambari-server/src/test/java/org/apache/ambari/server/stack/RepoUtilTest.java 
PRE-CREATION 
  
ambari-server/src/test/java/org/apache/ambari/server/stack/StackManagerCommonServicesTest.java
 1d73ff3 
  
ambari-server/src/test/java/org/apache/ambari/server/stack/StackModuleTest.java 
PRE-CREATION 
  
ambari-server/src/test/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartupTest.java
 PRE-CREATION 
  
ambari-server/src/test/resources/common-services/ADDON/1.0/configuration/addon-env.xml
 PRE-CREATION 
  ambari-server/src/test/resources/common-services/ADDON/1.0/metainfo.xml 
PRE-CREATION 
  
ambari-server/src/test/resources/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartupTest_initialRepos.json
 PRE-CREATION 
  
ambari-server/src/test/resources/stacks_with_common_services/HDP/0.2/services/ADDON/metainfo.xml
 PRE-CREATION 
  
ambari-server/src/test/resources/stacks_with_common_services/HDP/0.2/services/ADDON/repos/repoinfo.xml
 PRE-CREATION 
  
contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/configuration/microsoft-r-env.xml
 PRE-CREATION 
  
contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/package/scripts/microsoft_r.py
 61ea96b 
  
contrib/management-packs/microsoft-r_mpack/src/main/resources/custom-services/MICROSOFT_R/8.0.0/repos/repoinfo.xml
 PRE-CREATION 

Diff: https://reviews.apache.org/r/51238/diff/


Testing
---

- Manually tested
- Wrote new unit tests
- All unit tests passed except two which were failing in CI builds as well.


Thanks,

Balázs Bence Sári

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

2016-09-13 Thread Balázs Bence Sári



> On Sept. 7, 2016, 6:35 p.m., Nate Cole wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/state/RepositoryInfo.java,
> >  lines 166-169
> > 
> >
> > Objects.equals() for simplicity.

Fixed.


> On Sept. 7, 2016, 6:35 p.m., Nate Cole wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/state/RepositoryInfo.java,
> >  lines 179-181
> > 
> >
> > Use Objects.hash(...) for easier going

Fixed.


- Balázs Bence


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51238/#review148053
---


On Sept. 13, 2016, 9:25 a.m., Balázs Bence Sári wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51238/
> ---
> 
> (Updated Sept. 13, 2016, 9:25 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya, Nate Cole, Sumit Mohanty, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-15538
> https://issues.apache.org/jira/browse/AMBARI-15538
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Implementation contains the following things:
> - On startup, add-on service repos are loaded into the stack model
> - On startup, add-on service repos are merged into the VDF's downloaded by 
> LatestRepoCallable
> - On startup, if the is an existing cluster, it's repository version entity 
> is potentially updated with new add-on service repos.
> - Repository definitions contain two new optional fields: service_name and 
> service_version. (null for stack repositories) 
> - Small changes on the Mictrosoft-R mpack (supoorts HDP-2.4 and 2.5)
> - New unit tests
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java
>  7403dba 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  1d82928 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  bae4ab6 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProvider.java
>  02fc2ec 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/RepoUtil.java 
> PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java 
> bc94104 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java
>  3acc617 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
> 1fee589 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java
>  7bcd08b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartup.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/RepositoryInfo.java
>  83a 
>   ambari-server/src/main/java/org/apache/ambari/server/state/StackInfo.java 
> 14ff9de 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelper.java
>  d26b834 
>   ambari-server/src/main/resources/version_definition.xsd bd49028 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/RepoUtilTest.java 
> PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackManagerCommonServicesTest.java
>  1d73ff3 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackModuleTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartupTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/resources/common-services/ADDON/1.0/configuration/addon-env.xml
>  PRE-CREATION 
>   ambari-server/src/test/resources/common-services/ADDON/1.0/metainfo.xml 
> PRE-CREATION 
>   
> ambari-server/src/test/resources/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartupTest_initialRepos.json
>  PRE-CREATION 
>   
> ambari-server/src/test/resources/stacks_with_common_services/HDP/0.2/services/ADDON/metainfo.xml
>  PRE-CREATION 
>   
> ambari-server/src/test/resources/stacks_with_common_services/HDP/0.2/services/ADDON/repos/repoinfo.xml
>  PRE-CREATION 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/configuration/microsoft-r-env.xml
>  PRE-CREATION 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/package/scripts/microsoft_r.py
>  61ea96b 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/custom-services/MICROSOFT_R/8.0.0/repos/repoinfo.xml
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/51238/diff/
> 
> 
> Testing
> ---

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

2016-09-13 Thread Balázs Bence Sári


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51238/
---

(Updated Sept. 13, 2016, 9:25 a.m.)


Review request for Ambari, Jayush Luniya, Nate Cole, Sumit Mohanty, and 
Sebastian Toader.


Changes
---

Merged trunk and fixed stylistic issues.


Bugs: AMBARI-15538
https://issues.apache.org/jira/browse/AMBARI-15538


Repository: ambari


Description
---

Implementation contains the following things:
- On startup, add-on service repos are loaded into the stack model
- On startup, add-on service repos are merged into the VDF's downloaded by 
LatestRepoCallable
- On startup, if the is an existing cluster, it's repository version entity is 
potentially updated with new add-on service repos.
- Repository definitions contain two new optional fields: service_name and 
service_version. (null for stack repositories) 
- Small changes on the Mictrosoft-R mpack (supoorts HDP-2.4 and 2.5)
- New unit tests


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java
 7403dba 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
 1d82928 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
 bae4ab6 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProvider.java
 02fc2ec 
  ambari-server/src/main/java/org/apache/ambari/server/stack/RepoUtil.java 
PRE-CREATION 
  ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java 
bc94104 
  
ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java 
3acc617 
  ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
1fee589 
  
ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java
 7bcd08b 
  
ambari-server/src/main/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartup.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/state/RepositoryInfo.java 
83a 
  ambari-server/src/main/java/org/apache/ambari/server/state/StackInfo.java 
14ff9de 
  
ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelper.java
 d26b834 
  ambari-server/src/main/resources/version_definition.xsd bd49028 
  ambari-server/src/test/java/org/apache/ambari/server/stack/RepoUtilTest.java 
PRE-CREATION 
  
ambari-server/src/test/java/org/apache/ambari/server/stack/StackManagerCommonServicesTest.java
 1d73ff3 
  
ambari-server/src/test/java/org/apache/ambari/server/stack/StackModuleTest.java 
PRE-CREATION 
  
ambari-server/src/test/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartupTest.java
 PRE-CREATION 
  
ambari-server/src/test/resources/common-services/ADDON/1.0/configuration/addon-env.xml
 PRE-CREATION 
  ambari-server/src/test/resources/common-services/ADDON/1.0/metainfo.xml 
PRE-CREATION 
  
ambari-server/src/test/resources/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartupTest_initialRepos.json
 PRE-CREATION 
  
ambari-server/src/test/resources/stacks_with_common_services/HDP/0.2/services/ADDON/metainfo.xml
 PRE-CREATION 
  
ambari-server/src/test/resources/stacks_with_common_services/HDP/0.2/services/ADDON/repos/repoinfo.xml
 PRE-CREATION 
  
contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/configuration/microsoft-r-env.xml
 PRE-CREATION 
  
contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/package/scripts/microsoft_r.py
 61ea96b 
  
contrib/management-packs/microsoft-r_mpack/src/main/resources/custom-services/MICROSOFT_R/8.0.0/repos/repoinfo.xml
 PRE-CREATION 

Diff: https://reviews.apache.org/r/51238/diff/


Testing
---

- Manually tested
- Wrote new unit tests
- All unit tests passed except two which were failing in CI builds as well.


Thanks,

Balázs Bence Sári

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

2016-09-12 Thread Balázs Bence Sári



> On Sept. 7, 2016, 11:48 a.m., Tim Thorpe wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java,
> >  line 1133
> > 
> >
> > This won't include extension services.  You would need to loop through 
> > all the ServiceInfo objects and check to see if they have repos.

Added a new patch that handles it.


- Balázs Bence


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51238/#review148002
---


On Sept. 12, 2016, 8:48 p.m., Balázs Bence Sári wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51238/
> ---
> 
> (Updated Sept. 12, 2016, 8:48 p.m.)
> 
> 
> Review request for Ambari, Jayush Luniya, Nate Cole, Sumit Mohanty, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-15538
> https://issues.apache.org/jira/browse/AMBARI-15538
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Implementation contains the following things:
> - On startup, add-on service repos are loaded into the stack model
> - On startup, add-on service repos are merged into the VDF's downloaded by 
> LatestRepoCallable
> - On startup, if the is an existing cluster, it's repository version entity 
> is potentially updated with new add-on service repos.
> - Repository definitions contain two new optional fields: service_name and 
> service_version. (null for stack repositories) 
> - Small changes on the Mictrosoft-R mpack (supoorts HDP-2.4 and 2.5)
> - New unit tests
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java
>  7403dba 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  b1fd592 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  d20b1d7 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProvider.java
>  02fc2ec 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/RepoUtil.java 
> PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java 
> bc94104 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java
>  3acc617 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
> 23b0218 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java
>  7bcd08b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartup.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/RepositoryInfo.java
>  29776ed 
>   ambari-server/src/main/java/org/apache/ambari/server/state/StackInfo.java 
> 14ff9de 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelper.java
>  50d6028 
>   ambari-server/src/main/resources/version_definition.xsd bd49028 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/RepoUtilTest.java 
> PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackManagerCommonServicesTest.java
>  1d73ff3 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackModuleTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartupTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/resources/common-services/ADDON/1.0/configuration/addon-env.xml
>  PRE-CREATION 
>   ambari-server/src/test/resources/common-services/ADDON/1.0/metainfo.xml 
> PRE-CREATION 
>   
> ambari-server/src/test/resources/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartupTest_initialRepos.json
>  PRE-CREATION 
>   
> ambari-server/src/test/resources/stacks_with_common_services/HDP/0.2/services/ADDON/metainfo.xml
>  PRE-CREATION 
>   
> ambari-server/src/test/resources/stacks_with_common_services/HDP/0.2/services/ADDON/repos/repoinfo.xml
>  PRE-CREATION 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/configuration/microsoft-r-env.xml
>  PRE-CREATION 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/package/scripts/microsoft_r.py
>  61ea96b 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/custom-services/MICROSOFT_R/8.0.0/repos/repoinfo.xml
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/51238/diff/
> 
> 
> Testing
> ---
> 
> - Manually tested
> - Wrote new unit tests
> - All unit tests passed except two which were failing in CI builds as well.
> 
> 
> Thanks,
> 
> Balázs Bence

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

2016-09-12 Thread Balázs Bence Sári


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51238/
---

(Updated Sept. 12, 2016, 8:48 p.m.)


Review request for Ambari, Jayush Luniya, Nate Cole, Sumit Mohanty, and 
Sebastian Toader.


Changes
---

Fixed issues with stack inheritance and extension services.


Bugs: AMBARI-15538
https://issues.apache.org/jira/browse/AMBARI-15538


Repository: ambari


Description
---

Implementation contains the following things:
- On startup, add-on service repos are loaded into the stack model
- On startup, add-on service repos are merged into the VDF's downloaded by 
LatestRepoCallable
- On startup, if the is an existing cluster, it's repository version entity is 
potentially updated with new add-on service repos.
- Repository definitions contain two new optional fields: service_name and 
service_version. (null for stack repositories) 
- Small changes on the Mictrosoft-R mpack (supoorts HDP-2.4 and 2.5)
- New unit tests


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java
 7403dba 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
 b1fd592 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
 d20b1d7 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProvider.java
 02fc2ec 
  ambari-server/src/main/java/org/apache/ambari/server/stack/RepoUtil.java 
PRE-CREATION 
  ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java 
bc94104 
  
ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java 
3acc617 
  ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
23b0218 
  
ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java
 7bcd08b 
  
ambari-server/src/main/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartup.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/state/RepositoryInfo.java 
29776ed 
  ambari-server/src/main/java/org/apache/ambari/server/state/StackInfo.java 
14ff9de 
  
ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelper.java
 50d6028 
  ambari-server/src/main/resources/version_definition.xsd bd49028 
  ambari-server/src/test/java/org/apache/ambari/server/stack/RepoUtilTest.java 
PRE-CREATION 
  
ambari-server/src/test/java/org/apache/ambari/server/stack/StackManagerCommonServicesTest.java
 1d73ff3 
  
ambari-server/src/test/java/org/apache/ambari/server/stack/StackModuleTest.java 
PRE-CREATION 
  
ambari-server/src/test/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartupTest.java
 PRE-CREATION 
  
ambari-server/src/test/resources/common-services/ADDON/1.0/configuration/addon-env.xml
 PRE-CREATION 
  ambari-server/src/test/resources/common-services/ADDON/1.0/metainfo.xml 
PRE-CREATION 
  
ambari-server/src/test/resources/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartupTest_initialRepos.json
 PRE-CREATION 
  
ambari-server/src/test/resources/stacks_with_common_services/HDP/0.2/services/ADDON/metainfo.xml
 PRE-CREATION 
  
ambari-server/src/test/resources/stacks_with_common_services/HDP/0.2/services/ADDON/repos/repoinfo.xml
 PRE-CREATION 
  
contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/configuration/microsoft-r-env.xml
 PRE-CREATION 
  
contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/package/scripts/microsoft_r.py
 61ea96b 
  
contrib/management-packs/microsoft-r_mpack/src/main/resources/custom-services/MICROSOFT_R/8.0.0/repos/repoinfo.xml
 PRE-CREATION 

Diff: https://reviews.apache.org/r/51238/diff/


Testing
---

- Manually tested
- Wrote new unit tests
- All unit tests passed except two which were failing in CI builds as well.


Thanks,

Balázs Bence Sári

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

2016-09-12 Thread Balázs Bence Sári



> On Aug. 29, 2016, 5:54 p.m., Jayush Luniya wrote:
> > contrib/management-packs/microsoft-r_mpack/src/main/resources/custom-services/MICROSOFT_R/8.0.0/repos/repoinfo.xml,
> >  line 24
> > 
> >
> > A management pack can have one than one custom services. For example 
> > for HAWQ and PXF release artifacts for both the services are in the same 
> > repo.
> 
> Balázs Bence Sári wrote:
> Service name is removed

Also, if two services declare exactly the same repo (all fields are the same), 
the repo will be loaded and not treated as duplicate. If multiple 
extension/mpack services are delivered together and use the same repo, all 
services should declare the repo the same way.


- Balázs Bence


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51238/#review147167
---


On Sept. 5, 2016, 1:26 p.m., Balázs Bence Sári wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51238/
> ---
> 
> (Updated Sept. 5, 2016, 1:26 p.m.)
> 
> 
> Review request for Ambari, Jayush Luniya, Nate Cole, Sumit Mohanty, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-15538
> https://issues.apache.org/jira/browse/AMBARI-15538
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Implementation contains the following things:
> - On startup, add-on service repos are loaded into the stack model
> - On startup, add-on service repos are merged into the VDF's downloaded by 
> LatestRepoCallable
> - On startup, if the is an existing cluster, it's repository version entity 
> is potentially updated with new add-on service repos.
> - Repository definitions contain two new optional fields: service_name and 
> service_version. (null for stack repositories) 
> - Small changes on the Mictrosoft-R mpack (supoorts HDP-2.4 and 2.5)
> - New unit tests
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java
>  9f7419c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  b1fd592 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  d20b1d7 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProvider.java
>  02fc2ec 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/RepoUtil.java 
> PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java
>  3acc617 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
> 23b0218 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java
>  7bcd08b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartup.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/RepositoryInfo.java
>  29776ed 
>   ambari-server/src/main/java/org/apache/ambari/server/state/StackInfo.java 
> 14ff9de 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelper.java
>  50d6028 
>   ambari-server/src/main/resources/version_definition.xsd bd49028 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/RepoUtilTest.java 
> PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackManagerCommonServicesTest.java
>  1d73ff3 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackModuleTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartupTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/resources/common-services/ADDON/1.0/configuration/addon-env.xml
>  PRE-CREATION 
>   ambari-server/src/test/resources/common-services/ADDON/1.0/metainfo.xml 
> PRE-CREATION 
>   
> ambari-server/src/test/resources/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartupTest_initialRepos.json
>  PRE-CREATION 
>   
> ambari-server/src/test/resources/stacks_with_common_services/HDP/0.2/services/ADDON/metainfo.xml
>  PRE-CREATION 
>   
> ambari-server/src/test/resources/stacks_with_common_services/HDP/0.2/services/ADDON/repos/repoinfo.xml
>  PRE-CREATION 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/configuration/microsoft-r-env.xml
>  PRE-CREATION 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/package/scripts/microsoft_r.py
>  61ea96b 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/custom-services/MICROSOFT_R/8.0.0/repos/repoinfo.xml
>

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

2016-09-12 Thread Balázs Bence Sári



> On Aug. 29, 2016, 6:14 p.m., Jayush Luniya wrote:
> > contrib/management-packs/microsoft-r_mpack/src/main/resources/custom-services/MICROSOFT_R/8.0.0/repos/repoinfo.xml,
> >  line 24
> > 
> >
> > Is it mandatory to have servicename and serviceversion when defining a 
> > repoinfo at service-level? If, so can we add validations for it?
> 
> Balázs Bence Sári wrote:
> How about logginng a warning and not loading the repo in this case?

Deleted service_name and service_version


- Balázs Bence


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51238/#review147175
---


On Sept. 5, 2016, 1:26 p.m., Balázs Bence Sári wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51238/
> ---
> 
> (Updated Sept. 5, 2016, 1:26 p.m.)
> 
> 
> Review request for Ambari, Jayush Luniya, Nate Cole, Sumit Mohanty, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-15538
> https://issues.apache.org/jira/browse/AMBARI-15538
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Implementation contains the following things:
> - On startup, add-on service repos are loaded into the stack model
> - On startup, add-on service repos are merged into the VDF's downloaded by 
> LatestRepoCallable
> - On startup, if the is an existing cluster, it's repository version entity 
> is potentially updated with new add-on service repos.
> - Repository definitions contain two new optional fields: service_name and 
> service_version. (null for stack repositories) 
> - Small changes on the Mictrosoft-R mpack (supoorts HDP-2.4 and 2.5)
> - New unit tests
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java
>  9f7419c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  b1fd592 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  d20b1d7 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProvider.java
>  02fc2ec 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/RepoUtil.java 
> PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java
>  3acc617 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
> 23b0218 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java
>  7bcd08b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartup.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/RepositoryInfo.java
>  29776ed 
>   ambari-server/src/main/java/org/apache/ambari/server/state/StackInfo.java 
> 14ff9de 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelper.java
>  50d6028 
>   ambari-server/src/main/resources/version_definition.xsd bd49028 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/RepoUtilTest.java 
> PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackManagerCommonServicesTest.java
>  1d73ff3 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackModuleTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartupTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/resources/common-services/ADDON/1.0/configuration/addon-env.xml
>  PRE-CREATION 
>   ambari-server/src/test/resources/common-services/ADDON/1.0/metainfo.xml 
> PRE-CREATION 
>   
> ambari-server/src/test/resources/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartupTest_initialRepos.json
>  PRE-CREATION 
>   
> ambari-server/src/test/resources/stacks_with_common_services/HDP/0.2/services/ADDON/metainfo.xml
>  PRE-CREATION 
>   
> ambari-server/src/test/resources/stacks_with_common_services/HDP/0.2/services/ADDON/repos/repoinfo.xml
>  PRE-CREATION 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/configuration/microsoft-r-env.xml
>  PRE-CREATION 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/package/scripts/microsoft_r.py
>  61ea96b 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/custom-services/MICROSOFT_R/8.0.0/repos/repoinfo.xml
>  PRE-CREATION 
>   contrib/management-packs/microsoft-r_mpack/src/main/resources/mpack.json 
> f90ccce 
> 
> Diff: https://reviews.apache.org/r/51238/diff/
> 
> 
> Testing
> ---
> 
> - Manually tested

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

2016-09-12 Thread Balázs Bence Sári



> On Aug. 29, 2016, 5:27 p.m., Nate Cole wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProvider.java,
> >  line 562
> > 
> >
> > Weird that it's setOperatingSystems() but getting RepositoryInfos()

Renamed method to original name: serializeOperatingSystems()


- Balázs Bence


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51238/#review147153
---


On Sept. 5, 2016, 1:26 p.m., Balázs Bence Sári wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51238/
> ---
> 
> (Updated Sept. 5, 2016, 1:26 p.m.)
> 
> 
> Review request for Ambari, Jayush Luniya, Nate Cole, Sumit Mohanty, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-15538
> https://issues.apache.org/jira/browse/AMBARI-15538
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Implementation contains the following things:
> - On startup, add-on service repos are loaded into the stack model
> - On startup, add-on service repos are merged into the VDF's downloaded by 
> LatestRepoCallable
> - On startup, if the is an existing cluster, it's repository version entity 
> is potentially updated with new add-on service repos.
> - Repository definitions contain two new optional fields: service_name and 
> service_version. (null for stack repositories) 
> - Small changes on the Mictrosoft-R mpack (supoorts HDP-2.4 and 2.5)
> - New unit tests
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java
>  9f7419c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  b1fd592 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  d20b1d7 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProvider.java
>  02fc2ec 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/RepoUtil.java 
> PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java
>  3acc617 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
> 23b0218 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java
>  7bcd08b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartup.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/RepositoryInfo.java
>  29776ed 
>   ambari-server/src/main/java/org/apache/ambari/server/state/StackInfo.java 
> 14ff9de 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelper.java
>  50d6028 
>   ambari-server/src/main/resources/version_definition.xsd bd49028 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/RepoUtilTest.java 
> PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackManagerCommonServicesTest.java
>  1d73ff3 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackModuleTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartupTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/resources/common-services/ADDON/1.0/configuration/addon-env.xml
>  PRE-CREATION 
>   ambari-server/src/test/resources/common-services/ADDON/1.0/metainfo.xml 
> PRE-CREATION 
>   
> ambari-server/src/test/resources/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartupTest_initialRepos.json
>  PRE-CREATION 
>   
> ambari-server/src/test/resources/stacks_with_common_services/HDP/0.2/services/ADDON/metainfo.xml
>  PRE-CREATION 
>   
> ambari-server/src/test/resources/stacks_with_common_services/HDP/0.2/services/ADDON/repos/repoinfo.xml
>  PRE-CREATION 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/configuration/microsoft-r-env.xml
>  PRE-CREATION 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/package/scripts/microsoft_r.py
>  61ea96b 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/custom-services/MICROSOFT_R/8.0.0/repos/repoinfo.xml
>  PRE-CREATION 
>   contrib/management-packs/microsoft-r_mpack/src/main/resources/mpack.json 
> f90ccce 
> 
> Diff: https://reviews.apache.org/r/51238/diff/
> 
> 
> Testing
> ---
> 
> - Manually tested
> - Wrote new unit tests
> - All unit tests passed except two which were failing in CI builds as well.
> 
> 
> Thanks,
> 
> Balázs Bence Sári
> 
>

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

2016-09-07 Thread Nate Cole


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51238/#review148053
---




ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java
 (lines 39 - 40)


Syntax.



ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java
 (lines 45 - 46)


Odd syntax.  Usually see:

@Nullable
private String repoDir;



ambari-server/src/main/java/org/apache/ambari/server/state/RepositoryInfo.java 
(lines 166 - 169)


Objects.equals() for simplicity.



ambari-server/src/main/java/org/apache/ambari/server/state/RepositoryInfo.java 
(lines 179 - 181)


Use Objects.hash(...) for easier going


- Nate Cole


On Sept. 5, 2016, 9:26 a.m., Balázs Bence Sári wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51238/
> ---
> 
> (Updated Sept. 5, 2016, 9:26 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya, Nate Cole, Sumit Mohanty, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-15538
> https://issues.apache.org/jira/browse/AMBARI-15538
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Implementation contains the following things:
> - On startup, add-on service repos are loaded into the stack model
> - On startup, add-on service repos are merged into the VDF's downloaded by 
> LatestRepoCallable
> - On startup, if the is an existing cluster, it's repository version entity 
> is potentially updated with new add-on service repos.
> - Repository definitions contain two new optional fields: service_name and 
> service_version. (null for stack repositories) 
> - Small changes on the Mictrosoft-R mpack (supoorts HDP-2.4 and 2.5)
> - New unit tests
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java
>  9f7419c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  b1fd592 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  d20b1d7 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProvider.java
>  02fc2ec 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/RepoUtil.java 
> PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java
>  3acc617 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
> 23b0218 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java
>  7bcd08b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartup.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/RepositoryInfo.java
>  29776ed 
>   ambari-server/src/main/java/org/apache/ambari/server/state/StackInfo.java 
> 14ff9de 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelper.java
>  50d6028 
>   ambari-server/src/main/resources/version_definition.xsd bd49028 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/RepoUtilTest.java 
> PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackManagerCommonServicesTest.java
>  1d73ff3 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackModuleTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartupTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/resources/common-services/ADDON/1.0/configuration/addon-env.xml
>  PRE-CREATION 
>   ambari-server/src/test/resources/common-services/ADDON/1.0/metainfo.xml 
> PRE-CREATION 
>   
> ambari-server/src/test/resources/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartupTest_initialRepos.json
>  PRE-CREATION 
>   
> ambari-server/src/test/resources/stacks_with_common_services/HDP/0.2/services/ADDON/metainfo.xml
>  PRE-CREATION 
>   
> ambari-server/src/test/resources/stacks_with_common_services/HDP/0.2/services/ADDON/repos/repoinfo.xml
>  PRE-CREATION 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/configuration/microsoft-r-env.xml
>  PRE-CREATION 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/package/scripts/microsoft_r.py
>  61ea96b 
>   
>

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

2016-09-07 Thread Tim Thorpe


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51238/#review148002
---




ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
(line 1129)


This won't include extension services.  You would need to loop through all 
the ServiceInfo objects and check to see if they have repos.


- Tim Thorpe


On Sept. 5, 2016, 1:26 p.m., Balázs Bence Sári wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51238/
> ---
> 
> (Updated Sept. 5, 2016, 1:26 p.m.)
> 
> 
> Review request for Ambari, Jayush Luniya, Nate Cole, Sumit Mohanty, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-15538
> https://issues.apache.org/jira/browse/AMBARI-15538
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Implementation contains the following things:
> - On startup, add-on service repos are loaded into the stack model
> - On startup, add-on service repos are merged into the VDF's downloaded by 
> LatestRepoCallable
> - On startup, if the is an existing cluster, it's repository version entity 
> is potentially updated with new add-on service repos.
> - Repository definitions contain two new optional fields: service_name and 
> service_version. (null for stack repositories) 
> - Small changes on the Mictrosoft-R mpack (supoorts HDP-2.4 and 2.5)
> - New unit tests
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java
>  9f7419c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  b1fd592 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  d20b1d7 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProvider.java
>  02fc2ec 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/RepoUtil.java 
> PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java
>  3acc617 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
> 23b0218 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java
>  7bcd08b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartup.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/RepositoryInfo.java
>  29776ed 
>   ambari-server/src/main/java/org/apache/ambari/server/state/StackInfo.java 
> 14ff9de 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelper.java
>  50d6028 
>   ambari-server/src/main/resources/version_definition.xsd bd49028 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/RepoUtilTest.java 
> PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackManagerCommonServicesTest.java
>  1d73ff3 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackModuleTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartupTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/resources/common-services/ADDON/1.0/configuration/addon-env.xml
>  PRE-CREATION 
>   ambari-server/src/test/resources/common-services/ADDON/1.0/metainfo.xml 
> PRE-CREATION 
>   
> ambari-server/src/test/resources/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartupTest_initialRepos.json
>  PRE-CREATION 
>   
> ambari-server/src/test/resources/stacks_with_common_services/HDP/0.2/services/ADDON/metainfo.xml
>  PRE-CREATION 
>   
> ambari-server/src/test/resources/stacks_with_common_services/HDP/0.2/services/ADDON/repos/repoinfo.xml
>  PRE-CREATION 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/configuration/microsoft-r-env.xml
>  PRE-CREATION 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/package/scripts/microsoft_r.py
>  61ea96b 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/custom-services/MICROSOFT_R/8.0.0/repos/repoinfo.xml
>  PRE-CREATION 
>   contrib/management-packs/microsoft-r_mpack/src/main/resources/mpack.json 
> f90ccce 
> 
> Diff: https://reviews.apache.org/r/51238/diff/
> 
> 
> Testing
> ---
> 
> - Manually tested
> - Wrote new unit tests
> - All unit tests passed except two which were failing in CI builds as well.
> 
> 
> Thanks,
> 
> Balázs Bence Sári
> 
>

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

2016-09-05 Thread Balázs Bence Sári


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51238/
---

(Updated Sept. 5, 2016, 1:26 p.m.)


Review request for Ambari, Jayush Luniya, Nate Cole, Sumit Mohanty, and 
Sebastian Toader.


Changes
---

Review comments fixed


Bugs: AMBARI-15538
https://issues.apache.org/jira/browse/AMBARI-15538


Repository: ambari


Description
---

Implementation contains the following things:
- On startup, add-on service repos are loaded into the stack model
- On startup, add-on service repos are merged into the VDF's downloaded by 
LatestRepoCallable
- On startup, if the is an existing cluster, it's repository version entity is 
potentially updated with new add-on service repos.
- Repository definitions contain two new optional fields: service_name and 
service_version. (null for stack repositories) 
- Small changes on the Mictrosoft-R mpack (supoorts HDP-2.4 and 2.5)
- New unit tests


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java
 9f7419c 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
 b1fd592 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
 d20b1d7 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProvider.java
 02fc2ec 
  ambari-server/src/main/java/org/apache/ambari/server/stack/RepoUtil.java 
PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java 
3acc617 
  ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
23b0218 
  
ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java
 7bcd08b 
  
ambari-server/src/main/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartup.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/state/RepositoryInfo.java 
29776ed 
  ambari-server/src/main/java/org/apache/ambari/server/state/StackInfo.java 
14ff9de 
  
ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelper.java
 50d6028 
  ambari-server/src/main/resources/version_definition.xsd bd49028 
  ambari-server/src/test/java/org/apache/ambari/server/stack/RepoUtilTest.java 
PRE-CREATION 
  
ambari-server/src/test/java/org/apache/ambari/server/stack/StackManagerCommonServicesTest.java
 1d73ff3 
  
ambari-server/src/test/java/org/apache/ambari/server/stack/StackModuleTest.java 
PRE-CREATION 
  
ambari-server/src/test/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartupTest.java
 PRE-CREATION 
  
ambari-server/src/test/resources/common-services/ADDON/1.0/configuration/addon-env.xml
 PRE-CREATION 
  ambari-server/src/test/resources/common-services/ADDON/1.0/metainfo.xml 
PRE-CREATION 
  
ambari-server/src/test/resources/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartupTest_initialRepos.json
 PRE-CREATION 
  
ambari-server/src/test/resources/stacks_with_common_services/HDP/0.2/services/ADDON/metainfo.xml
 PRE-CREATION 
  
ambari-server/src/test/resources/stacks_with_common_services/HDP/0.2/services/ADDON/repos/repoinfo.xml
 PRE-CREATION 
  
contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/configuration/microsoft-r-env.xml
 PRE-CREATION 
  
contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/package/scripts/microsoft_r.py
 61ea96b 
  
contrib/management-packs/microsoft-r_mpack/src/main/resources/custom-services/MICROSOFT_R/8.0.0/repos/repoinfo.xml
 PRE-CREATION 
  contrib/management-packs/microsoft-r_mpack/src/main/resources/mpack.json 
f90ccce 

Diff: https://reviews.apache.org/r/51238/diff/


Testing
---

- Manually tested
- Wrote new unit tests
- All unit tests passed except two which were failing in CI builds as well.


Thanks,

Balázs Bence Sári

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

2016-09-02 Thread Balázs Bence Sári



> On Aug. 29, 2016, 5:54 p.m., Jayush Luniya wrote:
> > contrib/management-packs/microsoft-r_mpack/src/main/resources/custom-services/MICROSOFT_R/8.0.0/repos/repoinfo.xml,
> >  line 24
> > 
> >
> > A management pack can have one than one custom services. For example 
> > for HAWQ and PXF release artifacts for both the services are in the same 
> > repo.

Service name is removed


- Balázs Bence


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51238/#review147167
---


On Aug. 26, 2016, 9:25 a.m., Balázs Bence Sári wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51238/
> ---
> 
> (Updated Aug. 26, 2016, 9:25 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya, Nate Cole, Sumit Mohanty, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-15538
> https://issues.apache.org/jira/browse/AMBARI-15538
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Implementation contains the following things:
> - On startup, add-on service repos are loaded into the stack model
> - On startup, add-on service repos are merged into the VDF's downloaded by 
> LatestRepoCallable
> - On startup, if the is an existing cluster, it's repository version entity 
> is potentially updated with new add-on service repos.
> - Repository definitions contain two new optional fields: service_name and 
> service_version. (null for stack repositories) 
> - Small changes on the Mictrosoft-R mpack (supoorts HDP-2.4 and 2.5)
> - New unit tests
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java
>  92d47df 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java
>  36a2d99 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  63c99c6 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  097f01c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/RepositoryResponse.java
>  30bd0db 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RepositoryResourceProvider.java
>  3b5b0a7 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProvider.java
>  02fc2ec 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RepositoryEntity.java
>  49d53a9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RepositoryVersionEntity.java
>  25aa62b 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/RepoUtil.java 
> PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java
>  bfba021 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
> 0606f2a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java
>  7bcd08b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartup.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/RepositoryInfo.java
>  29776ed 
>   ambari-server/src/main/java/org/apache/ambari/server/state/StackInfo.java 
> 14ff9de 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/LatestRepoCallable.java
>  3c7c001 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/RepositoryXml.java
>  4a0ae3b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelper.java
>  50d6028 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java
>  3b5ed28 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog220.java
>  d806dde 
>   ambari-server/src/main/resources/version_definition.xsd bd49028 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/RepositoryResourceProviderTest.java
>  3a7b19b 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/RepoUtilTest.java 
> PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackManagerCommonServicesTest.java
>  1d73ff3 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackModuleTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartupTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelperTest.java
>  8d82d09 
>   
>

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

2016-09-01 Thread Balázs Bence Sári



> On Sept. 1, 2016, 2:10 p.m., Tim Thorpe wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/stack/RepoUtil.java, 
> > line 138
> > 
> >
> > The Optional class is only in JDK8. Currently Ambari is supposed to 
> > work with both JDK7 and 8.

This is Guava's Optional. I will fix the wildcard imports so it will be visible.


- Balázs Bence


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51238/#review147557
---


On Aug. 26, 2016, 9:25 a.m., Balázs Bence Sári wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51238/
> ---
> 
> (Updated Aug. 26, 2016, 9:25 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya, Nate Cole, Sumit Mohanty, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-15538
> https://issues.apache.org/jira/browse/AMBARI-15538
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Implementation contains the following things:
> - On startup, add-on service repos are loaded into the stack model
> - On startup, add-on service repos are merged into the VDF's downloaded by 
> LatestRepoCallable
> - On startup, if the is an existing cluster, it's repository version entity 
> is potentially updated with new add-on service repos.
> - Repository definitions contain two new optional fields: service_name and 
> service_version. (null for stack repositories) 
> - Small changes on the Mictrosoft-R mpack (supoorts HDP-2.4 and 2.5)
> - New unit tests
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java
>  92d47df 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java
>  36a2d99 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  63c99c6 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  097f01c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/RepositoryResponse.java
>  30bd0db 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RepositoryResourceProvider.java
>  3b5b0a7 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProvider.java
>  02fc2ec 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RepositoryEntity.java
>  49d53a9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RepositoryVersionEntity.java
>  25aa62b 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/RepoUtil.java 
> PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java
>  bfba021 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
> 0606f2a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java
>  7bcd08b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartup.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/RepositoryInfo.java
>  29776ed 
>   ambari-server/src/main/java/org/apache/ambari/server/state/StackInfo.java 
> 14ff9de 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/LatestRepoCallable.java
>  3c7c001 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/RepositoryXml.java
>  4a0ae3b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelper.java
>  50d6028 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java
>  3b5ed28 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog220.java
>  d806dde 
>   ambari-server/src/main/resources/version_definition.xsd bd49028 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/RepositoryResourceProviderTest.java
>  3a7b19b 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/RepoUtilTest.java 
> PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackManagerCommonServicesTest.java
>  1d73ff3 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackModuleTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartupTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelperTest.java
>  8d82d09 
>   
> ambari-server/src/test/resources/common-services/ADDON/1.0/configuration/addon-env.xml
>  PRE-CREATION 
>

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

2016-09-01 Thread Tim Thorpe



> On Aug. 29, 2016, 5:27 p.m., Nate Cole wrote:
> > I was under the impression that applying an m-pack would DIRECTLY update 
> > repoinfo.xml on the filesystem.  Has the design changed since then?
> > 
> > What is the purpose of carrying service version around with the repo?  When 
> > updating m-packs, that just means that there's that much more work to do to 
> > get it right.
> > 
> > How are you going to prevent an m-pack from overwriting an existing repo? 
> > (What if an m-pack names a repo "HDP-UTILS" or something - then they just 
> > blew over ours)?
> > 
> > You need to fix the code formatting so it's consistent and a bit easier to 
> > read.
> > 
> > In general, I don't like the idea of using service name to tie a repo to a 
> > service.  Maybe name this "reference" or a different indicator of where 
> > this repo came from.
> 
> Jayush Luniya wrote:
> RE: I was under the impression that applying an m-pack would DIRECTLY 
> update repoinfo.xml on the filesystem.
> Personally, it would be better if we can avoid updates to the stack's 
> repoinfo.xml and have the support to define repos at service level.
> 
> Balázs Bence Sári wrote:
> My implementation is based on this comment to the Jira (by Srimanth):
> 
> "Discussed with Nate Cole and Jayush Luniya, and the approach for 
> custom-services to specify their own repo location will be to provide a 
> /repos/repoinfo.xml inside the stack-version they will be in. This repo file 
> will be loaded by Ambari during startup into the 
> /api/v1/stacks/HDP/versions/2.4/repository_versions repos. Service repo files 
> have a restriction that their (repo-name, base-url) locations should be 
> unique and not conflict. When conflicts do occur, they will not be loaded 
> into the stacks model.
> Now the management-pack will provide such repos/ folder in 
> mpacks/custom-services/8.0.0/repos which will be linked into the stacks/ 
> folder.
> 
> ambari/ambari-server/src/main/resources/stacks/HDP/2.3/services/SERVICE_NAME/repos
>  -> mpacks/custom-services/8.0.0/repos"
> 
> Overwriting a stack repo is prohibited in the implementation.
> 
> service_name and service_version come from Srimanth, I can rename or 
> discard them. What should happen with them? Should I delete service_version 
> and rename service_name to reference? My two cents are that it's worth 
> keeping one to indicate that the repo does not come with the stack.
> 
> Nate Cole wrote:
> Association should be with the service itself (I think that's what Jayush 
> is saying), and we can access those as part of the stack endpoint for the 
> service, but should not be integrated with the stack definition.  We should 
> then include that repo info when we create the tasks to agents.
> 
> Balázs Bence Sári wrote:
> I think that's a bit different than how it was in the Jira comment (at 
> least to my interpetation) which I looked like a result of consensus. I can 
> implement it however. 
> 
> Also, we also need to make sure the user can overwrite those repo url's 
> when installing a cluster (Microsoft wants no public repo, repo locatinon in 
> the final mpack will be empty, corporates are expected to create their 
> private repos). With the current implementation, 
> api/v1/stacks/HDP/versions/2.3/operating_systems/redhat6 will return the 
> service repo along the stack repos and the user can override it on the UI. If 
> I move the repo to the service, should this API call still return it? 
> (guessing that UI uses this API to get the repos).
> 
> Nate Cole wrote:
> We can aggregate them in the API response, but I don't know that they 
> should be stored with the stack repos.

I added stack extensions 
(https://cwiki.apache.org/confluence/display/AMBARI/Extensions) for the purpose 
of allowing shared resources for a collection of custom services.  The way you 
are implementing service level repositories seems wrong.  If you have multiple 
services that require a custom repository, then you are randomly selecting 
which service's repoinfo.xml you will use.  If it really is a shared 
repository, it should be included at the extension level.

In general, I think we should be encouraging third parties to deliver their 
custom services in a consistent way (extensions installed with mpacks).


- Tim


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51238/#review147153
---


On Aug. 26, 2016, 9:25 a.m., Balázs Bence Sári wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51238/
> ---
> 
> (Updated Aug. 26, 2016, 9:25 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya, Nate Cole, Sumit Mohanty, and 
> Sebastian Toader.
>

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

2016-09-01 Thread Tim Thorpe


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51238/#review147557
---




ambari-server/src/main/java/org/apache/ambari/server/stack/RepoUtil.java (line 
138)


The Optional class is only in JDK8. Currently Ambari is supposed to work 
with both JDK7 and 8.


- Tim Thorpe


On Aug. 26, 2016, 9:25 a.m., Balázs Bence Sári wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51238/
> ---
> 
> (Updated Aug. 26, 2016, 9:25 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya, Nate Cole, Sumit Mohanty, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-15538
> https://issues.apache.org/jira/browse/AMBARI-15538
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Implementation contains the following things:
> - On startup, add-on service repos are loaded into the stack model
> - On startup, add-on service repos are merged into the VDF's downloaded by 
> LatestRepoCallable
> - On startup, if the is an existing cluster, it's repository version entity 
> is potentially updated with new add-on service repos.
> - Repository definitions contain two new optional fields: service_name and 
> service_version. (null for stack repositories) 
> - Small changes on the Mictrosoft-R mpack (supoorts HDP-2.4 and 2.5)
> - New unit tests
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java
>  92d47df 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java
>  36a2d99 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  63c99c6 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  097f01c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/RepositoryResponse.java
>  30bd0db 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RepositoryResourceProvider.java
>  3b5b0a7 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProvider.java
>  02fc2ec 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RepositoryEntity.java
>  49d53a9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RepositoryVersionEntity.java
>  25aa62b 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/RepoUtil.java 
> PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java
>  bfba021 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
> 0606f2a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java
>  7bcd08b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartup.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/RepositoryInfo.java
>  29776ed 
>   ambari-server/src/main/java/org/apache/ambari/server/state/StackInfo.java 
> 14ff9de 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/LatestRepoCallable.java
>  3c7c001 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/RepositoryXml.java
>  4a0ae3b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelper.java
>  50d6028 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java
>  3b5ed28 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog220.java
>  d806dde 
>   ambari-server/src/main/resources/version_definition.xsd bd49028 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/RepositoryResourceProviderTest.java
>  3a7b19b 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/RepoUtilTest.java 
> PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackManagerCommonServicesTest.java
>  1d73ff3 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackModuleTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartupTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelperTest.java
>  8d82d09 
>   
> ambari-server/src/test/resources/common-services/ADDON/1.0/configuration/addon-env.xml
>  PRE-CREATION 
>   ambari-server/src/test/resources/common-services/ADDON/1.0/metainfo.xml 
> PRE-CREATION 
>   
>

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

2016-08-30 Thread Nate Cole



> On Aug. 29, 2016, 1:27 p.m., Nate Cole wrote:
> > I was under the impression that applying an m-pack would DIRECTLY update 
> > repoinfo.xml on the filesystem.  Has the design changed since then?
> > 
> > What is the purpose of carrying service version around with the repo?  When 
> > updating m-packs, that just means that there's that much more work to do to 
> > get it right.
> > 
> > How are you going to prevent an m-pack from overwriting an existing repo? 
> > (What if an m-pack names a repo "HDP-UTILS" or something - then they just 
> > blew over ours)?
> > 
> > You need to fix the code formatting so it's consistent and a bit easier to 
> > read.
> > 
> > In general, I don't like the idea of using service name to tie a repo to a 
> > service.  Maybe name this "reference" or a different indicator of where 
> > this repo came from.
> 
> Jayush Luniya wrote:
> RE: I was under the impression that applying an m-pack would DIRECTLY 
> update repoinfo.xml on the filesystem.
> Personally, it would be better if we can avoid updates to the stack's 
> repoinfo.xml and have the support to define repos at service level.
> 
> Balázs Bence Sári wrote:
> My implementation is based on this comment to the Jira (by Srimanth):
> 
> "Discussed with Nate Cole and Jayush Luniya, and the approach for 
> custom-services to specify their own repo location will be to provide a 
> /repos/repoinfo.xml inside the stack-version they will be in. This repo file 
> will be loaded by Ambari during startup into the 
> /api/v1/stacks/HDP/versions/2.4/repository_versions repos. Service repo files 
> have a restriction that their (repo-name, base-url) locations should be 
> unique and not conflict. When conflicts do occur, they will not be loaded 
> into the stacks model.
> Now the management-pack will provide such repos/ folder in 
> mpacks/custom-services/8.0.0/repos which will be linked into the stacks/ 
> folder.
> 
> ambari/ambari-server/src/main/resources/stacks/HDP/2.3/services/SERVICE_NAME/repos
>  -> mpacks/custom-services/8.0.0/repos"
> 
> Overwriting a stack repo is prohibited in the implementation.
> 
> service_name and service_version come from Srimanth, I can rename or 
> discard them. What should happen with them? Should I delete service_version 
> and rename service_name to reference? My two cents are that it's worth 
> keeping one to indicate that the repo does not come with the stack.
> 
> Nate Cole wrote:
> Association should be with the service itself (I think that's what Jayush 
> is saying), and we can access those as part of the stack endpoint for the 
> service, but should not be integrated with the stack definition.  We should 
> then include that repo info when we create the tasks to agents.
> 
> Balázs Bence Sári wrote:
> I think that's a bit different than how it was in the Jira comment (at 
> least to my interpetation) which I looked like a result of consensus. I can 
> implement it however. 
> 
> Also, we also need to make sure the user can overwrite those repo url's 
> when installing a cluster (Microsoft wants no public repo, repo locatinon in 
> the final mpack will be empty, corporates are expected to create their 
> private repos). With the current implementation, 
> api/v1/stacks/HDP/versions/2.3/operating_systems/redhat6 will return the 
> service repo along the stack repos and the user can override it on the UI. If 
> I move the repo to the service, should this API call still return it? 
> (guessing that UI uses this API to get the repos).

We can aggregate them in the API response, but I don't know that they should be 
stored with the stack repos.


- Nate


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51238/#review147153
---


On Aug. 26, 2016, 5:25 a.m., Balázs Bence Sári wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51238/
> ---
> 
> (Updated Aug. 26, 2016, 5:25 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya, Nate Cole, Sumit Mohanty, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-15538
> https://issues.apache.org/jira/browse/AMBARI-15538
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Implementation contains the following things:
> - On startup, add-on service repos are loaded into the stack model
> - On startup, add-on service repos are merged into the VDF's downloaded by 
> LatestRepoCallable
> - On startup, if the is an existing cluster, it's repository version entity 
> is potentially updated with new add-on service repos.
> - Repository definitions contain two new optional fields: service_name and 
> service_version. (null for stack repositories) 
> - Small changes on the

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

2016-08-29 Thread Balázs Bence Sári



> On Aug. 29, 2016, 5:27 p.m., Nate Cole wrote:
> > I was under the impression that applying an m-pack would DIRECTLY update 
> > repoinfo.xml on the filesystem.  Has the design changed since then?
> > 
> > What is the purpose of carrying service version around with the repo?  When 
> > updating m-packs, that just means that there's that much more work to do to 
> > get it right.
> > 
> > How are you going to prevent an m-pack from overwriting an existing repo? 
> > (What if an m-pack names a repo "HDP-UTILS" or something - then they just 
> > blew over ours)?
> > 
> > You need to fix the code formatting so it's consistent and a bit easier to 
> > read.
> > 
> > In general, I don't like the idea of using service name to tie a repo to a 
> > service.  Maybe name this "reference" or a different indicator of where 
> > this repo came from.
> 
> Jayush Luniya wrote:
> RE: I was under the impression that applying an m-pack would DIRECTLY 
> update repoinfo.xml on the filesystem.
> Personally, it would be better if we can avoid updates to the stack's 
> repoinfo.xml and have the support to define repos at service level.
> 
> Balázs Bence Sári wrote:
> My implementation is based on this comment to the Jira (by Srimanth):
> 
> "Discussed with Nate Cole and Jayush Luniya, and the approach for 
> custom-services to specify their own repo location will be to provide a 
> /repos/repoinfo.xml inside the stack-version they will be in. This repo file 
> will be loaded by Ambari during startup into the 
> /api/v1/stacks/HDP/versions/2.4/repository_versions repos. Service repo files 
> have a restriction that their (repo-name, base-url) locations should be 
> unique and not conflict. When conflicts do occur, they will not be loaded 
> into the stacks model.
> Now the management-pack will provide such repos/ folder in 
> mpacks/custom-services/8.0.0/repos which will be linked into the stacks/ 
> folder.
> 
> ambari/ambari-server/src/main/resources/stacks/HDP/2.3/services/SERVICE_NAME/repos
>  -> mpacks/custom-services/8.0.0/repos"
> 
> Overwriting a stack repo is prohibited in the implementation.
> 
> service_name and service_version come from Srimanth, I can rename or 
> discard them. What should happen with them? Should I delete service_version 
> and rename service_name to reference? My two cents are that it's worth 
> keeping one to indicate that the repo does not come with the stack.
> 
> Nate Cole wrote:
> Association should be with the service itself (I think that's what Jayush 
> is saying), and we can access those as part of the stack endpoint for the 
> service, but should not be integrated with the stack definition.  We should 
> then include that repo info when we create the tasks to agents.

I think that's a bit different than how it was in the Jira comment (at least to 
my interpetation) which I looked like a result of consensus. I can implement it 
however. 

Also, we also need to make sure the user can overwrite those repo url's when 
installing a cluster (Microsoft wants no public repo, repo locatinon in the 
final mpack will be empty, corporates are expected to create their private 
repos). With the current implementation, 
api/v1/stacks/HDP/versions/2.3/operating_systems/redhat6 will return the 
service repo along the stack repos and the user can override it on the UI. If I 
move the repo to the service, should this API call still return it? (guessing 
that UI uses this API to get the repos).


- Balázs Bence


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51238/#review147153
---


On Aug. 26, 2016, 9:25 a.m., Balázs Bence Sári wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51238/
> ---
> 
> (Updated Aug. 26, 2016, 9:25 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya, Nate Cole, Sumit Mohanty, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-15538
> https://issues.apache.org/jira/browse/AMBARI-15538
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Implementation contains the following things:
> - On startup, add-on service repos are loaded into the stack model
> - On startup, add-on service repos are merged into the VDF's downloaded by 
> LatestRepoCallable
> - On startup, if the is an existing cluster, it's repository version entity 
> is potentially updated with new add-on service repos.
> - Repository definitions contain two new optional fields: service_name and 
> service_version. (null for stack repositories) 
> - Small changes on the Mictrosoft-R mpack (supoorts HDP-2.4 and 2.5)
> - New unit tests
> 
> 
> Diffs
> -
> 
>   
>

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

2016-08-29 Thread Balázs Bence Sári



> On Aug. 29, 2016, 6:14 p.m., Jayush Luniya wrote:
> > contrib/management-packs/microsoft-r_mpack/src/main/resources/custom-services/MICROSOFT_R/8.0.0/repos/repoinfo.xml,
> >  line 24
> > 
> >
> > Is it mandatory to have servicename and serviceversion when defining a 
> > repoinfo at service-level? If, so can we add validations for it?

How about logginng a warning and not loading the repo in this case?


- Balázs Bence


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51238/#review147175
---


On Aug. 26, 2016, 9:25 a.m., Balázs Bence Sári wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51238/
> ---
> 
> (Updated Aug. 26, 2016, 9:25 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya, Nate Cole, Sumit Mohanty, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-15538
> https://issues.apache.org/jira/browse/AMBARI-15538
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Implementation contains the following things:
> - On startup, add-on service repos are loaded into the stack model
> - On startup, add-on service repos are merged into the VDF's downloaded by 
> LatestRepoCallable
> - On startup, if the is an existing cluster, it's repository version entity 
> is potentially updated with new add-on service repos.
> - Repository definitions contain two new optional fields: service_name and 
> service_version. (null for stack repositories) 
> - Small changes on the Mictrosoft-R mpack (supoorts HDP-2.4 and 2.5)
> - New unit tests
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java
>  92d47df 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java
>  36a2d99 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  63c99c6 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  097f01c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/RepositoryResponse.java
>  30bd0db 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RepositoryResourceProvider.java
>  3b5b0a7 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProvider.java
>  02fc2ec 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RepositoryEntity.java
>  49d53a9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RepositoryVersionEntity.java
>  25aa62b 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/RepoUtil.java 
> PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java
>  bfba021 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
> 0606f2a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java
>  7bcd08b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartup.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/RepositoryInfo.java
>  29776ed 
>   ambari-server/src/main/java/org/apache/ambari/server/state/StackInfo.java 
> 14ff9de 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/LatestRepoCallable.java
>  3c7c001 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/RepositoryXml.java
>  4a0ae3b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelper.java
>  50d6028 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java
>  3b5ed28 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog220.java
>  d806dde 
>   ambari-server/src/main/resources/version_definition.xsd bd49028 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/RepositoryResourceProviderTest.java
>  3a7b19b 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/RepoUtilTest.java 
> PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackManagerCommonServicesTest.java
>  1d73ff3 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackModuleTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartupTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelperTest.java
>  8d82d09 
>   
>

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

2016-08-29 Thread Balázs Bence Sári



> On Aug. 29, 2016, 6:11 p.m., Jayush Luniya wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProvider.java,
> >  line 562
> > 
> >
> > Why do we need to change from serializeOperatingSystems to 
> > serializeRepositoryInfos? Thats odd

We have a new serializeOperatingSystems() with a different generic collection 
parameter. The new method is better described with the 
"serializeOperatingSystems" name as it works on operating systems. The two 
cannot coexist due to generic type erasure.


- Balázs Bence


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51238/#review147173
---


On Aug. 26, 2016, 9:25 a.m., Balázs Bence Sári wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51238/
> ---
> 
> (Updated Aug. 26, 2016, 9:25 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya, Nate Cole, Sumit Mohanty, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-15538
> https://issues.apache.org/jira/browse/AMBARI-15538
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Implementation contains the following things:
> - On startup, add-on service repos are loaded into the stack model
> - On startup, add-on service repos are merged into the VDF's downloaded by 
> LatestRepoCallable
> - On startup, if the is an existing cluster, it's repository version entity 
> is potentially updated with new add-on service repos.
> - Repository definitions contain two new optional fields: service_name and 
> service_version. (null for stack repositories) 
> - Small changes on the Mictrosoft-R mpack (supoorts HDP-2.4 and 2.5)
> - New unit tests
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java
>  92d47df 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java
>  36a2d99 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  63c99c6 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  097f01c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/RepositoryResponse.java
>  30bd0db 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RepositoryResourceProvider.java
>  3b5b0a7 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProvider.java
>  02fc2ec 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RepositoryEntity.java
>  49d53a9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RepositoryVersionEntity.java
>  25aa62b 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/RepoUtil.java 
> PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java
>  bfba021 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
> 0606f2a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java
>  7bcd08b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartup.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/RepositoryInfo.java
>  29776ed 
>   ambari-server/src/main/java/org/apache/ambari/server/state/StackInfo.java 
> 14ff9de 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/LatestRepoCallable.java
>  3c7c001 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/RepositoryXml.java
>  4a0ae3b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelper.java
>  50d6028 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java
>  3b5ed28 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog220.java
>  d806dde 
>   ambari-server/src/main/resources/version_definition.xsd bd49028 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/RepositoryResourceProviderTest.java
>  3a7b19b 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/RepoUtilTest.java 
> PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackManagerCommonServicesTest.java
>  1d73ff3 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackModuleTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartupTest.java
>  PRE-CREATION 
>   
>

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

2016-08-29 Thread Balázs Bence Sári



> On Aug. 29, 2016, 5:58 p.m., Jayush Luniya wrote:
> > A management pack for a custom service can be installed after a cluster is 
> > already deployed. Can you also test this scenario i.e. if installing a new 
> > management pack will it add this new repository for the cluster?

Yes, it is tested. UpdateActiveRepoVersionOnStartup takes care of it.


- Balázs Bence


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51238/#review147170
---


On Aug. 26, 2016, 9:25 a.m., Balázs Bence Sári wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51238/
> ---
> 
> (Updated Aug. 26, 2016, 9:25 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya, Nate Cole, Sumit Mohanty, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-15538
> https://issues.apache.org/jira/browse/AMBARI-15538
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Implementation contains the following things:
> - On startup, add-on service repos are loaded into the stack model
> - On startup, add-on service repos are merged into the VDF's downloaded by 
> LatestRepoCallable
> - On startup, if the is an existing cluster, it's repository version entity 
> is potentially updated with new add-on service repos.
> - Repository definitions contain two new optional fields: service_name and 
> service_version. (null for stack repositories) 
> - Small changes on the Mictrosoft-R mpack (supoorts HDP-2.4 and 2.5)
> - New unit tests
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java
>  92d47df 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java
>  36a2d99 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  63c99c6 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  097f01c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/RepositoryResponse.java
>  30bd0db 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RepositoryResourceProvider.java
>  3b5b0a7 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProvider.java
>  02fc2ec 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RepositoryEntity.java
>  49d53a9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RepositoryVersionEntity.java
>  25aa62b 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/RepoUtil.java 
> PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java
>  bfba021 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
> 0606f2a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java
>  7bcd08b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartup.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/RepositoryInfo.java
>  29776ed 
>   ambari-server/src/main/java/org/apache/ambari/server/state/StackInfo.java 
> 14ff9de 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/LatestRepoCallable.java
>  3c7c001 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/RepositoryXml.java
>  4a0ae3b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelper.java
>  50d6028 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java
>  3b5ed28 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog220.java
>  d806dde 
>   ambari-server/src/main/resources/version_definition.xsd bd49028 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/RepositoryResourceProviderTest.java
>  3a7b19b 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/RepoUtilTest.java 
> PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackManagerCommonServicesTest.java
>  1d73ff3 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackModuleTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartupTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelperTest.java
>  8d82d09 
>   
> ambari-server/src/test/resources/common-services/ADDON/1.0/configuration/addon-env.xml
>  PRE-CREATION 
>   ambari-server/src/test/resources/common-services/ADDON/1.0/metainfo.xml 
>

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

2016-08-29 Thread Balázs Bence Sári



> On Aug. 29, 2016, 5:34 p.m., Nate Cole wrote:
> > In addition, why not just keep repos with the service, then check for them 
> > when formulating the command.  Seems easier than munging everything else.

What exactly do you mean?


- Balázs Bence


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51238/#review147162
---


On Aug. 26, 2016, 9:25 a.m., Balázs Bence Sári wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51238/
> ---
> 
> (Updated Aug. 26, 2016, 9:25 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya, Nate Cole, Sumit Mohanty, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-15538
> https://issues.apache.org/jira/browse/AMBARI-15538
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Implementation contains the following things:
> - On startup, add-on service repos are loaded into the stack model
> - On startup, add-on service repos are merged into the VDF's downloaded by 
> LatestRepoCallable
> - On startup, if the is an existing cluster, it's repository version entity 
> is potentially updated with new add-on service repos.
> - Repository definitions contain two new optional fields: service_name and 
> service_version. (null for stack repositories) 
> - Small changes on the Mictrosoft-R mpack (supoorts HDP-2.4 and 2.5)
> - New unit tests
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java
>  92d47df 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java
>  36a2d99 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  63c99c6 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  097f01c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/RepositoryResponse.java
>  30bd0db 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RepositoryResourceProvider.java
>  3b5b0a7 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProvider.java
>  02fc2ec 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RepositoryEntity.java
>  49d53a9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RepositoryVersionEntity.java
>  25aa62b 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/RepoUtil.java 
> PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java
>  bfba021 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
> 0606f2a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java
>  7bcd08b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartup.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/RepositoryInfo.java
>  29776ed 
>   ambari-server/src/main/java/org/apache/ambari/server/state/StackInfo.java 
> 14ff9de 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/LatestRepoCallable.java
>  3c7c001 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/RepositoryXml.java
>  4a0ae3b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelper.java
>  50d6028 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java
>  3b5ed28 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog220.java
>  d806dde 
>   ambari-server/src/main/resources/version_definition.xsd bd49028 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/RepositoryResourceProviderTest.java
>  3a7b19b 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/RepoUtilTest.java 
> PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackManagerCommonServicesTest.java
>  1d73ff3 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackModuleTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartupTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelperTest.java
>  8d82d09 
>   
> ambari-server/src/test/resources/common-services/ADDON/1.0/configuration/addon-env.xml
>  PRE-CREATION 
>   ambari-server/src/test/resources/common-services/ADDON/1.0/metainfo.xml 
> PRE-CREATION 
>   
>

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

2016-08-29 Thread Balázs Bence Sári



> On Aug. 29, 2016, 5:27 p.m., Nate Cole wrote:
> > I was under the impression that applying an m-pack would DIRECTLY update 
> > repoinfo.xml on the filesystem.  Has the design changed since then?
> > 
> > What is the purpose of carrying service version around with the repo?  When 
> > updating m-packs, that just means that there's that much more work to do to 
> > get it right.
> > 
> > How are you going to prevent an m-pack from overwriting an existing repo? 
> > (What if an m-pack names a repo "HDP-UTILS" or something - then they just 
> > blew over ours)?
> > 
> > You need to fix the code formatting so it's consistent and a bit easier to 
> > read.
> > 
> > In general, I don't like the idea of using service name to tie a repo to a 
> > service.  Maybe name this "reference" or a different indicator of where 
> > this repo came from.
> 
> Jayush Luniya wrote:
> RE: I was under the impression that applying an m-pack would DIRECTLY 
> update repoinfo.xml on the filesystem.
> Personally, it would be better if we can avoid updates to the stack's 
> repoinfo.xml and have the support to define repos at service level.

My implementation is based on this comment to the Jira (by Srimanth):

"Discussed with Nate Cole and Jayush Luniya, and the approach for 
custom-services to specify their own repo location will be to provide a 
/repos/repoinfo.xml inside the stack-version they will be in. This repo file 
will be loaded by Ambari during startup into the 
/api/v1/stacks/HDP/versions/2.4/repository_versions repos. Service repo files 
have a restriction that their (repo-name, base-url) locations should be unique 
and not conflict. When conflicts do occur, they will not be loaded into the 
stacks model.
Now the management-pack will provide such repos/ folder in 
mpacks/custom-services/8.0.0/repos which will be linked into the stacks/ folder.
ambari/ambari-server/src/main/resources/stacks/HDP/2.3/services/SERVICE_NAME/repos
 -> mpacks/custom-services/8.0.0/repos"

Overwriting a stack repo is prohibited in the implementation.

service_name and service_version come from Srimanth, I can rename or discard 
them. What should happen with them? Should I delete service_version and rename 
service_name to reference? My two cents are that it's worth keeping one to 
indicate that the repo does not come with the stack.


> On Aug. 29, 2016, 5:27 p.m., Nate Cole wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/stack/RepoUtil.java, 
> > lines 20-21
> > 
> >
> > expand imports.  java/javax go first, then org, then com

Was due to wrong IntelliJ settings. Tried to fix all, but some escaped. Will 
fix it.


> On Aug. 29, 2016, 5:27 p.m., Nate Cole wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartup.java,
> >  line 122
> > 
> >
> > Why is the XML getting updated?  It makes it "unpure".  Updating the 
> > repos that are built FROM it should be sufficient.

Ok. (I just thought the XML and JSON representation should change together.)


- Balázs Bence


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51238/#review147153
---


On Aug. 26, 2016, 9:25 a.m., Balázs Bence Sári wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51238/
> ---
> 
> (Updated Aug. 26, 2016, 9:25 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya, Nate Cole, Sumit Mohanty, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-15538
> https://issues.apache.org/jira/browse/AMBARI-15538
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Implementation contains the following things:
> - On startup, add-on service repos are loaded into the stack model
> - On startup, add-on service repos are merged into the VDF's downloaded by 
> LatestRepoCallable
> - On startup, if the is an existing cluster, it's repository version entity 
> is potentially updated with new add-on service repos.
> - Repository definitions contain two new optional fields: service_name and 
> service_version. (null for stack repositories) 
> - Small changes on the Mictrosoft-R mpack (supoorts HDP-2.4 and 2.5)
> - New unit tests
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java
>  92d47df 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java
>  36a2d99 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  63c99c6 
>   
>

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

2016-08-29 Thread Jayush Luniya



> On Aug. 29, 2016, 5:27 p.m., Nate Cole wrote:
> > I was under the impression that applying an m-pack would DIRECTLY update 
> > repoinfo.xml on the filesystem.  Has the design changed since then?
> > 
> > What is the purpose of carrying service version around with the repo?  When 
> > updating m-packs, that just means that there's that much more work to do to 
> > get it right.
> > 
> > How are you going to prevent an m-pack from overwriting an existing repo? 
> > (What if an m-pack names a repo "HDP-UTILS" or something - then they just 
> > blew over ours)?
> > 
> > You need to fix the code formatting so it's consistent and a bit easier to 
> > read.
> > 
> > In general, I don't like the idea of using service name to tie a repo to a 
> > service.  Maybe name this "reference" or a different indicator of where 
> > this repo came from.

RE: I was under the impression that applying an m-pack would DIRECTLY update 
repoinfo.xml on the filesystem.
Personally, it would be better if we can avoid updates to the stack's 
repoinfo.xml and have the support to define repos at service level.


- Jayush


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51238/#review147153
---


On Aug. 26, 2016, 9:25 a.m., Balázs Bence Sári wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51238/
> ---
> 
> (Updated Aug. 26, 2016, 9:25 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya, Nate Cole, Sumit Mohanty, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-15538
> https://issues.apache.org/jira/browse/AMBARI-15538
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Implementation contains the following things:
> - On startup, add-on service repos are loaded into the stack model
> - On startup, add-on service repos are merged into the VDF's downloaded by 
> LatestRepoCallable
> - On startup, if the is an existing cluster, it's repository version entity 
> is potentially updated with new add-on service repos.
> - Repository definitions contain two new optional fields: service_name and 
> service_version. (null for stack repositories) 
> - Small changes on the Mictrosoft-R mpack (supoorts HDP-2.4 and 2.5)
> - New unit tests
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java
>  92d47df 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java
>  36a2d99 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  63c99c6 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  097f01c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/RepositoryResponse.java
>  30bd0db 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RepositoryResourceProvider.java
>  3b5b0a7 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProvider.java
>  02fc2ec 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RepositoryEntity.java
>  49d53a9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RepositoryVersionEntity.java
>  25aa62b 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/RepoUtil.java 
> PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java
>  bfba021 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
> 0606f2a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java
>  7bcd08b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartup.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/RepositoryInfo.java
>  29776ed 
>   ambari-server/src/main/java/org/apache/ambari/server/state/StackInfo.java 
> 14ff9de 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/LatestRepoCallable.java
>  3c7c001 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/RepositoryXml.java
>  4a0ae3b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelper.java
>  50d6028 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java
>  3b5ed28 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog220.java
>  d806dde 
>   ambari-server/src/main/resources/version_definition.xsd bd49028 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/RepositoryResourceProviderTest.java
>  3a7b19b 
>   
>

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

2016-08-29 Thread Jayush Luniya


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51238/#review147173
---




ambari-server/src/main/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProvider.java
 (line 562)


Why do we need to change from serializeOperatingSystems to 
serializeRepositoryInfos? Thats odd


- Jayush Luniya


On Aug. 26, 2016, 9:25 a.m., Balázs Bence Sári wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51238/
> ---
> 
> (Updated Aug. 26, 2016, 9:25 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya, Nate Cole, Sumit Mohanty, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-15538
> https://issues.apache.org/jira/browse/AMBARI-15538
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Implementation contains the following things:
> - On startup, add-on service repos are loaded into the stack model
> - On startup, add-on service repos are merged into the VDF's downloaded by 
> LatestRepoCallable
> - On startup, if the is an existing cluster, it's repository version entity 
> is potentially updated with new add-on service repos.
> - Repository definitions contain two new optional fields: service_name and 
> service_version. (null for stack repositories) 
> - Small changes on the Mictrosoft-R mpack (supoorts HDP-2.4 and 2.5)
> - New unit tests
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java
>  92d47df 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java
>  36a2d99 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  63c99c6 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  097f01c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/RepositoryResponse.java
>  30bd0db 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RepositoryResourceProvider.java
>  3b5b0a7 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProvider.java
>  02fc2ec 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RepositoryEntity.java
>  49d53a9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RepositoryVersionEntity.java
>  25aa62b 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/RepoUtil.java 
> PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java
>  bfba021 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
> 0606f2a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java
>  7bcd08b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartup.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/RepositoryInfo.java
>  29776ed 
>   ambari-server/src/main/java/org/apache/ambari/server/state/StackInfo.java 
> 14ff9de 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/LatestRepoCallable.java
>  3c7c001 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/RepositoryXml.java
>  4a0ae3b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelper.java
>  50d6028 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java
>  3b5ed28 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog220.java
>  d806dde 
>   ambari-server/src/main/resources/version_definition.xsd bd49028 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/RepositoryResourceProviderTest.java
>  3a7b19b 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/RepoUtilTest.java 
> PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackManagerCommonServicesTest.java
>  1d73ff3 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackModuleTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartupTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelperTest.java
>  8d82d09 
>   
> ambari-server/src/test/resources/common-services/ADDON/1.0/configuration/addon-env.xml
>  PRE-CREATION 
>   ambari-server/src/test/resources/common-services/ADDON/1.0/metainfo.xml 
> PRE-CREATION 
>   
>

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

2016-08-29 Thread Jayush Luniya


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51238/#review147170
---



A management pack for a custom service can be installed after a cluster is 
already deployed. Can you also test this scenario i.e. if installing a new 
management pack will it add this new repository for the cluster?

- Jayush Luniya


On Aug. 26, 2016, 9:25 a.m., Balázs Bence Sári wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51238/
> ---
> 
> (Updated Aug. 26, 2016, 9:25 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya, Nate Cole, Sumit Mohanty, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-15538
> https://issues.apache.org/jira/browse/AMBARI-15538
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Implementation contains the following things:
> - On startup, add-on service repos are loaded into the stack model
> - On startup, add-on service repos are merged into the VDF's downloaded by 
> LatestRepoCallable
> - On startup, if the is an existing cluster, it's repository version entity 
> is potentially updated with new add-on service repos.
> - Repository definitions contain two new optional fields: service_name and 
> service_version. (null for stack repositories) 
> - Small changes on the Mictrosoft-R mpack (supoorts HDP-2.4 and 2.5)
> - New unit tests
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java
>  92d47df 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java
>  36a2d99 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  63c99c6 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  097f01c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/RepositoryResponse.java
>  30bd0db 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RepositoryResourceProvider.java
>  3b5b0a7 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProvider.java
>  02fc2ec 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RepositoryEntity.java
>  49d53a9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RepositoryVersionEntity.java
>  25aa62b 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/RepoUtil.java 
> PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java
>  bfba021 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
> 0606f2a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java
>  7bcd08b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartup.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/RepositoryInfo.java
>  29776ed 
>   ambari-server/src/main/java/org/apache/ambari/server/state/StackInfo.java 
> 14ff9de 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/LatestRepoCallable.java
>  3c7c001 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/RepositoryXml.java
>  4a0ae3b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelper.java
>  50d6028 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java
>  3b5ed28 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog220.java
>  d806dde 
>   ambari-server/src/main/resources/version_definition.xsd bd49028 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/RepositoryResourceProviderTest.java
>  3a7b19b 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/RepoUtilTest.java 
> PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackManagerCommonServicesTest.java
>  1d73ff3 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackModuleTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartupTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelperTest.java
>  8d82d09 
>   
> ambari-server/src/test/resources/common-services/ADDON/1.0/configuration/addon-env.xml
>  PRE-CREATION 
>   ambari-server/src/test/resources/common-services/ADDON/1.0/metainfo.xml 
> PRE-CREATION 
>   
>

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

2016-08-29 Thread Jayush Luniya


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51238/#review147167
---




contrib/management-packs/microsoft-r_mpack/src/main/resources/custom-services/MICROSOFT_R/8.0.0/repos/repoinfo.xml
 (line 24)


A management pack can have one than one custom services. For example for 
HAWQ and PXF release artifacts for both the services are in the same repo.


- Jayush Luniya


On Aug. 26, 2016, 9:25 a.m., Balázs Bence Sári wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51238/
> ---
> 
> (Updated Aug. 26, 2016, 9:25 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya, Nate Cole, Sumit Mohanty, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-15538
> https://issues.apache.org/jira/browse/AMBARI-15538
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Implementation contains the following things:
> - On startup, add-on service repos are loaded into the stack model
> - On startup, add-on service repos are merged into the VDF's downloaded by 
> LatestRepoCallable
> - On startup, if the is an existing cluster, it's repository version entity 
> is potentially updated with new add-on service repos.
> - Repository definitions contain two new optional fields: service_name and 
> service_version. (null for stack repositories) 
> - Small changes on the Mictrosoft-R mpack (supoorts HDP-2.4 and 2.5)
> - New unit tests
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java
>  92d47df 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java
>  36a2d99 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  63c99c6 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  097f01c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/RepositoryResponse.java
>  30bd0db 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RepositoryResourceProvider.java
>  3b5b0a7 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProvider.java
>  02fc2ec 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RepositoryEntity.java
>  49d53a9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RepositoryVersionEntity.java
>  25aa62b 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/RepoUtil.java 
> PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java
>  bfba021 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
> 0606f2a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java
>  7bcd08b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartup.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/RepositoryInfo.java
>  29776ed 
>   ambari-server/src/main/java/org/apache/ambari/server/state/StackInfo.java 
> 14ff9de 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/LatestRepoCallable.java
>  3c7c001 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/RepositoryXml.java
>  4a0ae3b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelper.java
>  50d6028 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java
>  3b5ed28 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog220.java
>  d806dde 
>   ambari-server/src/main/resources/version_definition.xsd bd49028 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/RepositoryResourceProviderTest.java
>  3a7b19b 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/RepoUtilTest.java 
> PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackManagerCommonServicesTest.java
>  1d73ff3 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackModuleTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartupTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelperTest.java
>  8d82d09 
>   
> ambari-server/src/test/resources/common-services/ADDON/1.0/configuration/addon-env.xml
>  PRE-CREATION 
>   ambari-server/src/test/resources/common-services/ADDON/1.0/metainfo.xml 
> PRE-CREATION 
>   
>

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

2016-08-29 Thread Nate Cole


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51238/#review147162
---



In addition, why not just keep repos with the service, then check for them when 
formulating the command.  Seems easier than munging everything else.

- Nate Cole


On Aug. 26, 2016, 5:25 a.m., Balázs Bence Sári wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51238/
> ---
> 
> (Updated Aug. 26, 2016, 5:25 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya, Nate Cole, Sumit Mohanty, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-15538
> https://issues.apache.org/jira/browse/AMBARI-15538
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Implementation contains the following things:
> - On startup, add-on service repos are loaded into the stack model
> - On startup, add-on service repos are merged into the VDF's downloaded by 
> LatestRepoCallable
> - On startup, if the is an existing cluster, it's repository version entity 
> is potentially updated with new add-on service repos.
> - Repository definitions contain two new optional fields: service_name and 
> service_version. (null for stack repositories) 
> - Small changes on the Mictrosoft-R mpack (supoorts HDP-2.4 and 2.5)
> - New unit tests
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java
>  92d47df 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java
>  36a2d99 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  63c99c6 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  097f01c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/RepositoryResponse.java
>  30bd0db 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RepositoryResourceProvider.java
>  3b5b0a7 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProvider.java
>  02fc2ec 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RepositoryEntity.java
>  49d53a9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RepositoryVersionEntity.java
>  25aa62b 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/RepoUtil.java 
> PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java
>  bfba021 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
> 0606f2a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java
>  7bcd08b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartup.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/RepositoryInfo.java
>  29776ed 
>   ambari-server/src/main/java/org/apache/ambari/server/state/StackInfo.java 
> 14ff9de 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/LatestRepoCallable.java
>  3c7c001 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/RepositoryXml.java
>  4a0ae3b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelper.java
>  50d6028 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java
>  3b5ed28 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog220.java
>  d806dde 
>   ambari-server/src/main/resources/version_definition.xsd bd49028 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/RepositoryResourceProviderTest.java
>  3a7b19b 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/RepoUtilTest.java 
> PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackManagerCommonServicesTest.java
>  1d73ff3 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackModuleTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartupTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelperTest.java
>  8d82d09 
>   
> ambari-server/src/test/resources/common-services/ADDON/1.0/configuration/addon-env.xml
>  PRE-CREATION 
>   ambari-server/src/test/resources/common-services/ADDON/1.0/metainfo.xml 
> PRE-CREATION 
>   
> ambari-server/src/test/resources/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartupTest_initialRepos.json
>  PRE-CREATION 
>   
>

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

2016-08-29 Thread Nate Cole


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51238/#review147153
---



I was under the impression that applying an m-pack would DIRECTLY update 
repoinfo.xml on the filesystem.  Has the design changed since then?

What is the purpose of carrying service version around with the repo?  When 
updating m-packs, that just means that there's that much more work to do to get 
it right.

How are you going to prevent an m-pack from overwriting an existing repo? (What 
if an m-pack names a repo "HDP-UTILS" or something - then they just blew over 
ours)?

You need to fix the code formatting so it's consistent and a bit easier to read.

In general, I don't like the idea of using service name to tie a repo to a 
service.  Maybe name this "reference" or a different indicator of where this 
repo came from.


ambari-server/src/main/java/org/apache/ambari/server/controller/RepositoryResponse.java
 (lines 168 - 183)


Formatting.



ambari-server/src/main/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProvider.java
 (line 562)


Weird that it's setOperatingSystems() but getting RepositoryInfos()



ambari-server/src/main/java/org/apache/ambari/server/stack/RepoUtil.java (lines 
20 - 21)


expand imports.  java/javax go first, then org, then com



ambari-server/src/main/java/org/apache/ambari/server/stack/RepoUtil.java (line 
95)


Javadoc



ambari-server/src/main/java/org/apache/ambari/server/stack/RepoUtil.java (line 
110)


Javadoc



ambari-server/src/main/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartup.java
 (line 122)


Why is the XML getting updated?  It makes it "unpure".  Updating the repos 
that are built FROM it should be sufficient.



ambari-server/src/main/java/org/apache/ambari/server/state/RepositoryInfo.java 
(lines 21 - 23)


Ordering



ambari-server/src/main/java/org/apache/ambari/server/state/RepositoryInfo.java 
(lines 213 - 215)


formatting.


- Nate Cole


On Aug. 26, 2016, 5:25 a.m., Balázs Bence Sári wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51238/
> ---
> 
> (Updated Aug. 26, 2016, 5:25 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya, Nate Cole, Sumit Mohanty, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-15538
> https://issues.apache.org/jira/browse/AMBARI-15538
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Implementation contains the following things:
> - On startup, add-on service repos are loaded into the stack model
> - On startup, add-on service repos are merged into the VDF's downloaded by 
> LatestRepoCallable
> - On startup, if the is an existing cluster, it's repository version entity 
> is potentially updated with new add-on service repos.
> - Repository definitions contain two new optional fields: service_name and 
> service_version. (null for stack repositories) 
> - Small changes on the Mictrosoft-R mpack (supoorts HDP-2.4 and 2.5)
> - New unit tests
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java
>  92d47df 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java
>  36a2d99 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  63c99c6 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  097f01c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/RepositoryResponse.java
>  30bd0db 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RepositoryResourceProvider.java
>  3b5b0a7 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProvider.java
>  02fc2ec 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RepositoryEntity.java
>  49d53a9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RepositoryVersionEntity.java
>  25aa62b 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/RepoUtil.java 
> PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java
>  bfba021 
>

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services

31 matches

Site Navigation

Mail list logo

Footer information