> On March 2, 2016, 9:37 p.m., jun aoki wrote: > > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/service_advisor_HDP206HBASE.py, > > line 1 > > <https://reviews.apache.org/r/44210/diff/1/?file=1275516#file1275516line1> > > > > Hi Tim, nice effort to decouple stack_advisor! > > > > Is this file naming service_advisor_HDP206HBASE.py a bit redundant? > > this is under > > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE, any files > > under this folder is meant for HDP, version 2.0.6 and HBASE. > > Tim Thorpe wrote: > I tried having the name as you stated but had a problem with python. For > example in 2.3/services/HDFS, I have the following: > > SCRIPT_DIR = os.path.dirname(os.path.abspath(file)) > PARENT_DIR = os.path.join(SCRIPT_DIR, '../../../2.2/services/HDFS/') > sys.path.append(PARENT_DIR) > from service_advisor_HDP22HDFS import * > > If the file name is the same in 2.3 and 2.2, then "from service_advisor > import *" could reference the 2.2 or the 2.3 file. I tried it with import > HDP22HDFSServiceAdvisor as well and both of them failed to find the correct > parent class. > > The other way I could have solved this is to load all the service advisor > files from the default_stack_advisor.py walking my way up the stack. This > would have allowed you to avoid the inheritance code in the > service_advisor.py files but it would restrict you from inheriting from > outside of your stack (unless at that point you started including the > inheritance code and the change of file names). > > jun aoki wrote: > I believe it has to treated as python packages so that file path matters. > ``` > sys.path.append("..../stacks") > from HDP.2_2.services.HDFS import service_advisor # you'd probably have > to deal with "." in versions. > ```
I have figured out a solution for this. So the naming of the service advisor file would always be service_advisor.py without being prefaced with the stack name, version and service name. - Tim ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/44210/#review121720 ----------------------------------------------------------- On March 1, 2016, 4:53 p.m., Tim Thorpe wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/44210/ > ----------------------------------------------------------- > > (Updated March 1, 2016, 4:53 p.m.) > > > Review request for Ambari, Sumit Mohanty and Yusaku Sako. > > > Bugs: AMBARI-15226 > https://issues.apache.org/jira/browse/AMBARI-15226 > > > Repository: ambari > > > Description > ------- > > Currently the stack advisor is defined under each stack version such as > HDP/2.3. The problem with this is that it restricts the services that can be > added to the stack. If a custom service is to be added, they would need to > modify the stack advisor. If the configuration recommendation and validation > can be done at the service level then the custom service could just include > their own recommendations and validations separately. > > > Diffs > ----- > > ambari-agent/pom.xml c2c993f > ambari-server/src/main/assemblies/server.xml e1a4919 > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py > PRE-CREATION > > ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py > PRE-CREATION > ambari-server/src/main/resources/scripts/stack_advisor.py cdd9acb > > ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/GANGLIA/service_advisor_BIGTOP08GANGLIA.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HBASE/service_advisor_BIGTOP08HBASE.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HDFS/service_advisor_BIGTOP08HDFS.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HIVE/service_advisor_BIGTOP08HIVE.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/OOZIE/service_advisor_BIGTOP08OOZIE.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/YARN/service_advisor_BIGTOP08YARN.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/ZOOKEEPER/service_advisor_BIGTOP08ZOOKEEPER.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/stack_advisor.py > 53591cd > > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/AMBARI_METRICS/service_advisor_HDP206AMBARI_METRICS.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/GANGLIA/service_advisor_HDP206GANGLIA.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/service_advisor_HDP206HBASE.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/service_advisor_HDP206HDFS.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HIVE/service_advisor_HDP206HIVE.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/OOZIE/service_advisor_HDP206OOZIE.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/YARN/service_advisor_HDP206YARN.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/ZOOKEEPER/service_advisor_HDP206ZOOKEEPER.py > PRE-CREATION > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py > f6f8cde > > ambari-server/src/main/resources/stacks/HDP/2.1/services/FALCON/service_advisor_HDP21FALCON.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.1/services/HIVE/service_advisor_HDP21HIVE.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.1/services/OOZIE/service_advisor_HDP21OOZIE.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/service_advisor_HDP21STORM.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.1/services/TEZ/service_advisor_HDP21TEZ.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.1/services/YARN/service_advisor_HDP21YARN.py > PRE-CREATION > ambari-server/src/main/resources/stacks/HDP/2.1/services/stack_advisor.py > 3a552b8 > > ambari-server/src/main/resources/stacks/HDP/2.2/services/AMBARI_METRICS/metainfo.xml > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.2/services/AMBARI_METRICS/service_advisor_HDP22AMBARI_METRICS.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.2/services/HBASE/service_advisor_HDP22HBASE.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/service_advisor_HDP22HDFS.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.2/services/HIVE/service_advisor_HDP22HIVE.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.2/services/KAFKA/service_advisor_HDP22KAFKA.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.2/services/KNOX/service_advisor_HDP22KNOX.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.2/services/RANGER/service_advisor_HDP22RANGER.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.2/services/STORM/service_advisor_HDP22STORM.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.2/services/TEZ/service_advisor_HDP22TEZ.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.2/services/YARN/service_advisor_HDP22YARN.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.3.GlusterFS/services/stack_advisor.py > bffbc26 > > ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/service_advisor_HDP23HBASE.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/service_advisor_HDP23HDFS.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.3/services/HIVE/service_advisor_HDP23HIVE.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.3/services/KAFKA/service_advisor_HDP23KAFKA.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER/service_advisor_HDP23RANGER.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER_KMS/service_advisor_HDP23RANGER_KMS.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.3/services/SPARK/service_advisor_HDP23SPARK.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.3/services/TEZ/service_advisor_HDP23TEZ.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.3/services/YARN/service_advisor_HDP23YARN.py > PRE-CREATION > ambari-server/src/main/resources/stacks/HDP/2.4/services/stack_advisor.py > 985c101 > ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py > 9dd5cc7 > ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py > af269de > > ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/AMBARI_METRICS/service_advisor_HDPWIN21AMBARI_METRICS.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/FALCON/service_advisor_HDPWIN21FALCON.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HBASE/service_advisor_HDPWIN21HBASE.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HDFS/service_advisor_HDPWIN21HDFS.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HIVE/service_advisor_HDPWIN21HIVE.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/OOZIE/service_advisor_HDPWIN21OOZIE.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/STORM/service_advisor_HDPWIN21STORM.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/TEZ/service_advisor_HDPWIN21TEZ.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/YARN/service_advisor_HDPWIN21YARN.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/ZOOKEEPER/service_advisor_HDPWIN21ZOOKEEPER.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/stack_advisor.py > b99c484 > > ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/AMBARI_METRICS/service_advisor_HDPWIN22AMBARI_METRICS.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HBASE/service_advisor_HDPWIN22HBASE.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HDFS/service_advisor_HDPWIN22HDFS.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HIVE/service_advisor_HDPWIN22HIVE.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/KNOX/service_advisor_HDPWIN22KNOX.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/STORM/service_advisor_HDPWIN22STORM.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/TEZ/service_advisor_HDPWIN22TEZ.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/YARN/service_advisor_HDPWIN22YARN.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HBASE/service_advisor_HDPWIN23HBASE.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HDFS/service_advisor_HDPWIN23HDFS.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HIVE/service_advisor_HDPWIN23HIVE.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/TEZ/service_advisor_HDPWIN23TEZ.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/YARN/service_advisor_HDPWIN23YARN.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/stack_advisor.py > 3631c88 > ambari-server/src/main/resources/stacks/default_stack_advisor.py > PRE-CREATION > ambari-server/src/main/resources/stacks/stack_advisor.py 539bd25 > ambari-server/src/test/python/TestStackAdvisor.py 149ae1d > ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py > e15582e > ambari-server/src/test/python/stacks/2.1/common/test_stack_advisor.py > 7ede5a3 > ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py > d95e6c7 > ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor_perf.py > 5b8c30c > ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py > 8932bde > > ambari-server/src/test/resources/stacks/XYZ/1.0.0/services/YARN/service_advisor_XYZ100YARN.py > PRE-CREATION > ambari-server/src/test/resources/stacks/XYZ/1.0.0/services/stack_advisor.py > ba140bb > > ambari-server/src/test/resources/stacks/XYZ/1.0.1/services/YARN/service_advisor_XYZ101YARN.py > PRE-CREATION > ambari-server/src/test/resources/stacks/XYZ/1.0.1/services/stack_advisor.py > 74a4b31 > ambari-server/src/test/resources/stacks/old/HDP206stack_advisor.py > PRE-CREATION > ambari-server/src/test/resources/stacks/old/HDP21stack_advisor.py > PRE-CREATION > ambari-server/src/test/resources/stacks/old/HDP22stack_advisor.py > PRE-CREATION > ambari-server/src/test/resources/stacks/old/HDP23stack_advisor.py > PRE-CREATION > ambari-server/src/test/resources/stacks/old/HDP24stack_advisor.py > PRE-CREATION > ambari-server/src/test/resources/stacks/old/HDP25stack_advisor.py > PRE-CREATION > ambari-server/src/test/resources/stacks/old/HDP26stack_advisor.py > PRE-CREATION > ambari-server/src/test/resources/stacks/old/stack_advisor.py PRE-CREATION > > Diff: https://reviews.apache.org/r/44210/diff/ > > > Testing > ------- > > Ran all the non java unit tests. > > Total run:921 > Total errors:1 > Total failures:0 > > The only test which failed also failed on a fresh trunk test run: test_start > (test_metrics_grafana.TestMetricsGrafana) > > > Thanks, > > Tim Thorpe > >