-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44210/#review127197
-----------------------------------------------------------



Tim, thanks for contributing to the stack-advisor feature.

When we started the stack-advisor feature, there was a lot of interconnect 
between base service component-layout and configurations (HDFS, YARN, 
MapReduce, Hive, HBase) where common calculations were used by multiple 
services. To maximise readability and reuse of code, we took the 1 
stack-advisor/stack-version approach. This has kept the number of scripts 
managable (8 scripts for all versions of HDP stack) to maintain and debug when 
problems occur. It is very useful for stacks where a lot of interconnected 
services exist and stack-advisor should account for all the usecases.

Now, coming to this JIRA, it is undeniable that individual service folders 
should also be able to contribute to stack-advisor logic - it is useful when 
the service is not so dependent on other services and can independently provide 
its component-layout and configurations. Also, it is makes it easier to 
contribute a rich service by dropping a folder into the stack.

With that requirement said, I think a smaller change would be to provide a 
mechanism by which a service can contribute logic to the stack-version's 
stack-advisor. This would allow for both worlds of highly-interconnected and 
highly-independent services to coexist in the stack-advisor framework. For an 
independent service, it can provide a stack-advisor script which only dealt 
with it's component-layout and configuration - that can be overlaid on top of 
the stack-version's recommendations/validations.

My vote would be to have such an optional service-level stack-advisor script, 
which can be provided only when needed. We can discuss its design and how best 
to implement it.

- Srimanth Gunturi


On March 1, 2016, 4:53 p.m., Tim Thorpe wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44210/
> -----------------------------------------------------------
> 
> (Updated March 1, 2016, 4:53 p.m.)
> 
> 
> Review request for Ambari, Sumit Mohanty and Yusaku Sako.
> 
> 
> Bugs: AMBARI-15226
>     https://issues.apache.org/jira/browse/AMBARI-15226
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> Currently the stack advisor is defined under each stack version such as 
> HDP/2.3. The problem with this is that it restricts the services that can be 
> added to the stack. If a custom service is to be added, they would need to 
> modify the stack advisor. If the configuration recommendation and validation 
> can be done at the service level then the custom service could just include 
> their own recommendations and validations separately.
> 
> 
> Diffs
> -----
> 
>   ambari-agent/pom.xml c2c993f 
>   ambari-server/src/main/assemblies/server.xml e1a4919 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py 
> PRE-CREATION 
>   ambari-server/src/main/resources/scripts/stack_advisor.py cdd9acb 
>   
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/GANGLIA/service_advisor_BIGTOP08GANGLIA.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HBASE/service_advisor_BIGTOP08HBASE.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HDFS/service_advisor_BIGTOP08HDFS.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HIVE/service_advisor_BIGTOP08HIVE.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/OOZIE/service_advisor_BIGTOP08OOZIE.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/YARN/service_advisor_BIGTOP08YARN.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/ZOOKEEPER/service_advisor_BIGTOP08ZOOKEEPER.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/stack_advisor.py 
> 53591cd 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/AMBARI_METRICS/service_advisor_HDP206AMBARI_METRICS.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/GANGLIA/service_advisor_HDP206GANGLIA.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/service_advisor_HDP206HBASE.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/service_advisor_HDP206HDFS.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HIVE/service_advisor_HDP206HIVE.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/OOZIE/service_advisor_HDP206OOZIE.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/YARN/service_advisor_HDP206YARN.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/ZOOKEEPER/service_advisor_HDP206ZOOKEEPER.py
>  PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
> f6f8cde 
>   
> ambari-server/src/main/resources/stacks/HDP/2.1/services/FALCON/service_advisor_HDP21FALCON.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.1/services/HIVE/service_advisor_HDP21HIVE.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.1/services/OOZIE/service_advisor_HDP21OOZIE.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/service_advisor_HDP21STORM.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.1/services/TEZ/service_advisor_HDP21TEZ.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.1/services/YARN/service_advisor_HDP21YARN.py
>  PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.1/services/stack_advisor.py 
> 3a552b8 
>   
> ambari-server/src/main/resources/stacks/HDP/2.2/services/AMBARI_METRICS/metainfo.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.2/services/AMBARI_METRICS/service_advisor_HDP22AMBARI_METRICS.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HBASE/service_advisor_HDP22HBASE.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/service_advisor_HDP22HDFS.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HIVE/service_advisor_HDP22HIVE.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.2/services/KAFKA/service_advisor_HDP22KAFKA.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.2/services/KNOX/service_advisor_HDP22KNOX.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.2/services/RANGER/service_advisor_HDP22RANGER.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.2/services/STORM/service_advisor_HDP22STORM.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.2/services/TEZ/service_advisor_HDP22TEZ.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.2/services/YARN/service_advisor_HDP22YARN.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3.GlusterFS/services/stack_advisor.py
>  bffbc26 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/service_advisor_HDP23HBASE.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/service_advisor_HDP23HDFS.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HIVE/service_advisor_HDP23HIVE.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/KAFKA/service_advisor_HDP23KAFKA.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER/service_advisor_HDP23RANGER.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER_KMS/service_advisor_HDP23RANGER_KMS.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/SPARK/service_advisor_HDP23SPARK.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/TEZ/service_advisor_HDP23TEZ.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/YARN/service_advisor_HDP23YARN.py
>  PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.4/services/stack_advisor.py 
> 985c101 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
> 9dd5cc7 
>   ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py 
> af269de 
>   
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/AMBARI_METRICS/service_advisor_HDPWIN21AMBARI_METRICS.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/FALCON/service_advisor_HDPWIN21FALCON.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HBASE/service_advisor_HDPWIN21HBASE.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HDFS/service_advisor_HDPWIN21HDFS.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HIVE/service_advisor_HDPWIN21HIVE.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/OOZIE/service_advisor_HDPWIN21OOZIE.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/STORM/service_advisor_HDPWIN21STORM.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/TEZ/service_advisor_HDPWIN21TEZ.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/YARN/service_advisor_HDPWIN21YARN.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/ZOOKEEPER/service_advisor_HDPWIN21ZOOKEEPER.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/stack_advisor.py 
> b99c484 
>   
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/AMBARI_METRICS/service_advisor_HDPWIN22AMBARI_METRICS.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HBASE/service_advisor_HDPWIN22HBASE.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HDFS/service_advisor_HDPWIN22HDFS.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HIVE/service_advisor_HDPWIN22HIVE.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/KNOX/service_advisor_HDPWIN22KNOX.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/STORM/service_advisor_HDPWIN22STORM.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/TEZ/service_advisor_HDPWIN22TEZ.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/YARN/service_advisor_HDPWIN22YARN.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HBASE/service_advisor_HDPWIN23HBASE.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HDFS/service_advisor_HDPWIN23HDFS.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HIVE/service_advisor_HDPWIN23HIVE.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/TEZ/service_advisor_HDPWIN23TEZ.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/YARN/service_advisor_HDPWIN23YARN.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/stack_advisor.py 
> 3631c88 
>   ambari-server/src/main/resources/stacks/default_stack_advisor.py 
> PRE-CREATION 
>   ambari-server/src/main/resources/stacks/stack_advisor.py 539bd25 
>   ambari-server/src/test/python/TestStackAdvisor.py 149ae1d 
>   ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py 
> e15582e 
>   ambari-server/src/test/python/stacks/2.1/common/test_stack_advisor.py 
> 7ede5a3 
>   ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py 
> d95e6c7 
>   ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor_perf.py 
> 5b8c30c 
>   ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 
> 8932bde 
>   
> ambari-server/src/test/resources/stacks/XYZ/1.0.0/services/YARN/service_advisor_XYZ100YARN.py
>  PRE-CREATION 
>   ambari-server/src/test/resources/stacks/XYZ/1.0.0/services/stack_advisor.py 
> ba140bb 
>   
> ambari-server/src/test/resources/stacks/XYZ/1.0.1/services/YARN/service_advisor_XYZ101YARN.py
>  PRE-CREATION 
>   ambari-server/src/test/resources/stacks/XYZ/1.0.1/services/stack_advisor.py 
> 74a4b31 
>   ambari-server/src/test/resources/stacks/old/HDP206stack_advisor.py 
> PRE-CREATION 
>   ambari-server/src/test/resources/stacks/old/HDP21stack_advisor.py 
> PRE-CREATION 
>   ambari-server/src/test/resources/stacks/old/HDP22stack_advisor.py 
> PRE-CREATION 
>   ambari-server/src/test/resources/stacks/old/HDP23stack_advisor.py 
> PRE-CREATION 
>   ambari-server/src/test/resources/stacks/old/HDP24stack_advisor.py 
> PRE-CREATION 
>   ambari-server/src/test/resources/stacks/old/HDP25stack_advisor.py 
> PRE-CREATION 
>   ambari-server/src/test/resources/stacks/old/HDP26stack_advisor.py 
> PRE-CREATION 
>   ambari-server/src/test/resources/stacks/old/stack_advisor.py PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/44210/diff/
> 
> 
> Testing
> -------
> 
> Ran all the non java unit tests.  
> 
> Total run:921
> Total errors:1
> Total failures:0
> 
> The only test which failed also failed on a fresh trunk test run: test_start 
> (test_metrics_grafana.TestMetricsGrafana)
> 
> 
> Thanks,
> 
> Tim Thorpe
> 
>

Reply via email to