[ https://issues.apache.org/jira/browse/KNOX-1014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16163460#comment-16163460 ]
Phil Zampino edited comment on KNOX-1014 at 9/12/17 6:42 PM: ------------------------------------------------------------- KNOX-1014.patch provides the foundation for service discovery and topology generation. After applying the patch, if you have access to an Ambari cluster, you can try the following: # Locate or [create|https://cwiki.apache.org/confluence/display/AMBARI/Quick+Start+Guide] an Ambari cluster # Extract the <gateway> element and its contents from conf/topologies/sandbox.xml, and save it in conf/shared-providers/sandbox-providers.xml # Provision the username/password for your Ambari instance (bin/knoxcli.sh create-alias AMBARI_CLUSTER_ADMIN_NAME --value AMBARI_CLUSTER_ADMIN_PASSWORD) # Create a simple descriptor (JSON): Replace YOUR_AMBARI_HOST(e.g., c6401.ambari.apache.org), AMBARI_CLUSTER_ADMIN_NAME, YOUR_CLUSTER_NAME with the appropriate values {noformat} { "discovery-type":"AMBARI", "discovery-address":"http://YOUR_AMBARI_HOST:8080", "discovery-user":"AMBARI_CLUSTER_ADMIN_NAME", "provider-config-ref":"sandbox-providers.xml", "cluster":"YOUR_CLUSTER_NAME", "services":[ {"name":"NAMENODE"}, {"name":"JOBTRACKER"}, {"name":"WEBHDFS"}, {"name":"WEBHCAT"}, {"name":"OOZIE"}, {"name":"WEBHBASE"}, {"name":"HIVE"}, {"name":"RESOURCEMANAGER"}, {"name":"AMBARI", "url":"http://c6401.ambari.apache.org:8080"}, {"name":"AMBARIUI", "url":"http://c6401.ambari.apache.org:8080"} ] } {noformat} # Place this simple descriptor in conf/descriptors/YOUR_DESCRIPTOR_NAME.json # Allow the TopologyService to notice the simple descriptor, generate and deploy the full topology. # Check conf/topologies, and notice YOUR_DESCRIPTOR_NAME.xml there # cat conf/topologies/YOUR_DESCRIPTOR_NAME.xml to see that the service URLs have been populated from the Ambari cluster details # Verify the WEBHDFS service URL was correctly discovered curl -ivku guest:guest-password https://localhost:8443/gateway/YOUR_DESCRIPTOR_NAME/webhdfs/v1/tmp?op=LISTSTATUS Now, you can try adding/modifying/removing combinations of provider configurations and descriptions to see the results. * If you modify a descriptor, it will update the full topology so the changes are reflected. * If you modify a shared provider configuration, it will update any referencing descriptors, which will in turn update the associated full topology files. * If you delete a simple descriptor, the associated topology will be removed/undeployed. * If you delete a topology file, the associated simple descriptor will also be deleted. * If you delete a shared provider configuration, any referencing descriptors (and associated topology files) will also be deleted, and the topologies will be undeployed. was (Author: pzampino): KNOX-1014.patch provides the foundation for service discovery and topology generation. After applying the patch, if you have access to an Ambari cluster, you can try the following: # Locate or [create|https://cwiki.apache.org/confluence/display/AMBARI/Quick+Start+Guide] an Ambari cluster # Extract the <gateway> element and its contents from conf/topologies/sandbox.xml, and save it in conf/shared-providers/sandbox-providers.xml # Provision the username/password for your Ambari instance (bin/knoxcli.sh create-alias AMBARI_CLUSTER_ADMIN_NAME --value AMBARI_CLUSTER_ADMIN_PASSWORD) # Create a simple descriptor (JSON): Replace YOUR_AMBARI_HOST(e.g., c6401.ambari.apache.org), AMBARI_CLUSTER_ADMIN_NAME, YOUR_CLUSTER_NAME with the appropriate values { "discovery-type":"AMBARI", "discovery-address":"http://YOUR_AMBARI_HOST:8080", "discovery-user":"AMBARI_CLUSTER_ADMIN_NAME", "provider-config-ref":"sandbox-providers.xml", "cluster":"YOUR_CLUSTER_NAME", "services":[ {"name":"NAMENODE"}, {"name":"JOBTRACKER"}, {"name":"WEBHDFS"}, {"name":"WEBHCAT"}, {"name":"OOZIE"}, {"name":"WEBHBASE"}, {"name":"HIVE"}, {"name":"RESOURCEMANAGER"}, {"name":"AMBARI", "url":"http://c6401.ambari.apache.org:8080"}, {"name":"AMBARIUI", "url":"http://c6401.ambari.apache.org:8080"} ] } # Place this simple descriptor in conf/descriptors/YOUR_DESCRIPTOR_NAME.json # Allow the TopologyService to notice the simple descriptor, generate and deploy the full topology. # Check conf/topologies, and notice YOUR_DESCRIPTOR_NAME.xml there # cat conf/topologies/YOUR_DESCRIPTOR_NAME.xml to see that the service URLs have been populated from the Ambari cluster details # Verify the WEBHDFS service URL was correctly discovered curl -ivku guest:guest-password https://localhost:8443/gateway/YOUR_DESCRIPTOR_NAME/webhdfs/v1/tmp?op=LISTSTATUS Now, you can try adding/modifying/removing combinations of provider configurations and descriptions to see the results. * If you modify a descriptor, it will update the full topology so the changes are reflected. * If you modify a shared provider configuration, it will update any referencing descriptors, which will in turn update the associated full topology files. * If you delete a simple descriptor, the associated topology will be removed/undeployed. * If you delete a topology file, the associated simple descriptor will also be deleted. * If you delete a shared provider configuration, any referencing descriptors (and associated topology files) will also be deleted, and the topologies will be undeployed. > Service Discovery and Topology Generation Framework > --------------------------------------------------- > > Key: KNOX-1014 > URL: https://issues.apache.org/jira/browse/KNOX-1014 > Project: Apache Knox > Issue Type: Sub-task > Components: Server > Reporter: Phil Zampino > Assignee: Phil Zampino > Labels: kip-8 > Attachments: KNOX-1014.patch > > > Implement the foundation for Service Discovery and Topology Generation. > * Define simple descriptor format (YAML, JSON, properties, etc...) > * Local simple descriptor discovery (re-use existing FileAlterationMonitor?) > * Ambari service discovery (REST API interactions and model construction) > ** Configuration > *** How to plug-in discovery implementations (service loader?) > *** How to configure authentication (credentials/trust) with the service > registries > * Topology assembly from simple descriptor and discovery details > * Topology deployment (something more than copy to the conf/topologies > directory?) -- This message was sent by Atlassian JIRA (v6.4.14#64029)