Inline ... ________________________________ From: Christopher Jackson <[email protected]> Sent: Sunday, May 10, 2015 8:49 AM To: [email protected] Cc: Sumit Mohanty; Alejandro Fernandez Subject: Re: Ambari Custom Service Questions.
Thanks for this information. I have a few follow up questions asked inline. Thanks. Regards, Christopher Jackson On May 9, 2015, at 9:25 PM, Sumit Mohanty <[email protected]<mailto:[email protected]>> wrote: One addition to question 4. ________________________________ From: Alejandro Fernandez <[email protected]<mailto:[email protected]>> Sent: Saturday, May 09, 2015 1:44 PM To: [email protected]<mailto:[email protected]>; Christopher Jackson Subject: Re: Ambari Custom Service Questions. Hi Christopher, these are all very good questions, and it would be useful to supplement the wiki with them. Comments inline. On 5/9/15, 11:59 AM, "Christopher Jackson" <[email protected]<mailto:[email protected]>> wrote: Hi All, I’ve been in the process of creating a custom Ambari service over the past week and have quite a few general questions in which I haven’t found answers for in documentation or on the wiki. I was hoping some of you could help answer any of the following questions. Thanks in advance. 1) I’ve noticed that when restarting a services component that is of type ‘CLIENT’ that its install and configure method are invoked. I’m wondering if it’s in intended and if so why? For components of type ‘MASTER’ a restart doesn’t seem to invoke install and configure again, it just invokes stop then start. I ask about this because in my custom service I have a CLIENT component in which there are some steps I do in the install stage that I don’t want repeated every time its restarted. Alejandro> For clients, a "Restart" or "Refresh Configs" essentially only needs to make sure that the client libraries are present and the configs are setup. Since technically a client cannot be restarted because it is not a daemon, the code is written in such a way that it is idempotent, so no harm in installing libs that are already present, or settings configs that are already there. For Masters, they have independent commands to Install the libs, and Restart the daemon process. For your client, is there any artifact you can check to avoid running your one-time-install multiple times? Chris> Thank you for the explanation. Yes I can work around this by checking if certain artifacts exist. 2) Can someone explain the implication of the auto-deploy and its child elements in the context where the following snippet would be placed in the metainfo.xml file of a custom service component (MY_SERVICE/MY_COMPONENT_MASTER)? <dependency> <name>HBASE/HBASE_CLIENT</name> <scope>host</scope> <auto-deploy> <enabled>true</enabled> <co-locate>MY_SERVICE/MY_COMPONENT_CLIENT</co-locate> </auto-deploy> </dependency> Alejandro> This is a really good question. Some components depend on others, and those dependencies need to be installed automatically, either anywhere in the cluster, the same host, or on the same host that contains another component. You probably don't need the "co-locate" tag, since it is used to indicate masters that must be together, and is configured in the UI during the Service Install Wizard. Chris> As part of my custom service I am adding libraries to both the HDFS and HBASE Services. My custom services Client component ensures these libraries get installed on the system and symlinked to the appropriate lib folder. Here is my concern. If a user installs a cluster with my service and then later adds an additional node to the cluster with HDFS or HBASE installed on that node how can I ensure that my custom services client component also gets installed? Is there a way to do that without defining a custom stack or modifying the HBASE and HDFS Service definitions? Sumit> At this point user needs to explicitly install the custom service client as well. The capability that is needed is for a component to specify in its own metainfo that it is a mandatory dependency to another component. Feel free to create a JIRA for this feature. 3) When creating a configuration file for a custom service what are the valid entries for a <property-type> tag? PASSWORD, and what else? Is there any other child elements of ‘property’ that are useful? Perhaps anything that allows you to provide a regular expression for validation? Alejandro> These are PASSWORD, USER, GROUP, TEXT. These only contain name, value, description. If you need to use regex to validate a property, that means the UI should take care of it, so take a look at ambari-web module, particularly, config_property_helper.js 4) Is there some function to restart a service in resource_management or other ambari python library? Or should I just be restarting services using the command line tools and ensuring to update the appropriate pid files? I ask this question because I’ve noticed I cannot restart a service using the Ambari API as part of the installation/configuration steps of my custom service, as the restart commands are queued while the custom service installation/configuration is running, and will cause a timeout. I’m looking for a solution to this problem, what’s recommended if not one of the approaches I’ve asked about above? Alejandro> Once the service is defined in the metainfo.xml file, along with the python file to use, it's up to that script to decide how to install and restart your service. Ambari doesn't couple config changes with forcing the service to restart automatically, this is because if a user makes a config change or installs something, Ambari only highlights that the service needs to be restarted, but it's up to the user to decide when to do it. If you wanted to do automatic restarts upon config changes, then that your python script would then have to call the restart() method. Take a look at script.py Sumit> In general the pattern of calling Ambari Server APIs from the implementation of install/configure/start of a component definition is not supported. This is because only one command can be executed at anytime on a host. In theory, you could make the call from the install/configure/start implementation and not wait for the call to complete. Can you explain the scenario a bit more? Are you restarting your custom service or some other service from the install/configuration of the custom service. Chris> As part of my custom service I alter configurations for HDFS, HBASE, OOZIE, ZOOKEEPER, MAPREDUCE2, and YARN. I add libraries to HDFS, HBASE, and OOZIE server (Symlink libraries from my component into the specified services lib folder). To add the libraries to OOZIE server I’m stopping it, executing the oozie-setup shell script and then restarting it. So to make sure these services pick up the new configurations, additional libraries I wanted to restart them. Is there a better way to handle adding the libraries for these services? Sumit> There are two parts to this - a) host level changes and b) Ambari/cluster level operations. Host level changes is done by the install/configure implementation of the CUSTOM-SERVICE-CLIENT. The rest of the operations need Ambari API calls. What you have is correct - as in doing the lib installation etc. as part of CLIENT install. Its the calls to Ambari APIs from the install script that is non-standard (I assume thats what you have now). Is there a way you can call the APIs from outside of Ambari - when user adds hosts or adds service? Perhaps a capability is needed where UI allows customizing of what happens when host is added or service is added. You can still call the APIs from CLIENT install/start scripts if you do not wait for completion. 5) How can I allow for the removal of a custom service from the ambari console? I know there is a sequence of Ambari API commands I can run to: stop the service, stop each of its components, and then finally delete the service but is there any way to incorporate that into the Ambari UI instead of having a script on the filesystem that invokes those calls? Alejandro> Ambari UI allows deleting a component as long as it is stopped. This is done from the Host Details page. But deleting a service entirely is more complicated and I don't believe the API supports this today (probably because it would have do delete records from a lot of tables and is not a common use case). Thanks so much, Christopher Jackson
