You are correct, in 3.0 the Log Feeder configuration is not editable as an Ambari property. The json genereated by the file input.config-<service_name>.json.j2 will bootstrap this configuration handled by Log Search internally, after it was parsed by any of the Log Feeders for the first time, later it will have no longer any effect if it was modified. This configuration can be edited later only on the Log Search UI on a new window specifically for editing configurations, which is not ready yet, or via calling the REST API of the Log Search server directly. We’ll have some documentation detailing how to handle configurations as soon as the UI is finished.
> The mapping among Ambari components and LogIds which used to be a property, > is happening now based on the service metainfo.xml and the definition of > LogIds in that file. is that correct? Yes, that’s correct. From: Juanjo Marron <[email protected]<mailto:[email protected]>> Reply-To: Juanjo Marron <[email protected]<mailto:[email protected]>> Date: Tuesday, October 24, 2017 at 11:53 PM To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>>, default <[email protected]<mailto:[email protected]>> Subject: Re: 3rd party services requiring to restart Log Search OK. I Understand now. Thanks for the info Miklos So these templates input.config-<service_name>.json.j2 are just the content property in the <service-name>-logsearch-conf.xml, right? The use of templates seems similar to the old design of Log Search integration before <service-name>-logsearch-conf.xml came into the game, but avoiding to edit the HadoopServiceConfig.json.j2 file. That's great! Are these templates still exposed in the Ambari UI as they were before as part of the <service-name>-logsearch-conf.xml file? I saw <service-name>-logsearch-conf.xml files are now removed from services definition. So I assume user's cant not edit the templates directly in the UI. If they are edited at file level, do they get copied over to Log Feeder config directory. Hook will be just work after the install. The wont be updated if there are later changes, right? The mapping among Ambari components and LogIds which used to be a property, is happening now based on the service metainfo.xml and the definition of LogIds in that file. is that correct? Sorry for all the questions but we are currently worting on the integration of 3rd party services into Log Search and I could not details in the JIRA or documentation explaining the future changes. Thanks again On Tuesday, October 24, 2017 11:24 AM, Miklos Gergely <[email protected]<mailto:[email protected]>> wrote: Yes, if they have a file called input.config-<service_name>.json.j2 at package/templates in their stack definition, and they are installed with Ambari. There is a post install hook in Ambari looking for such files, and if it is found in the stack definition it is copied into the Log Feeder config directory, even if Log Feeder is not even installed yet. It is a jinja file where you can reference configuration properties, e.g. log directory settings. Check out Zookeeper’s configuration for example: https://github.com/apache/ambari/blob/trunk/ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5/package/templates/input.config-zookeeper.json.j2 From: Juanjo Marron <[email protected]<mailto:[email protected]><mailto:[email protected]<mailto:[email protected]>>> Reply-To: Juanjo Marron <[email protected]<mailto:[email protected]><mailto:[email protected]<mailto:[email protected]>>> Date: Tuesday, October 24, 2017 at 7:18 PM To: default <[email protected]<mailto:[email protected]><mailto:[email protected]<mailto:[email protected]>>>, "[email protected]<mailto:[email protected]><mailto:[email protected]<mailto:[email protected]>>" <[email protected]<mailto:[email protected]><mailto:[email protected]<mailto:[email protected]>>> Subject: Re: 3rd party services requiring to restart Log Search Thanks Miklos for your quick response and the details provided. True, I was taking a look just to trunk code. I see now all the *-logsearch-conf.xml files listed as <configuration-dependencies> for the HDP stack services in branch-2.5, which explains the required restart. A 3rd party service won't be able to modify Log Seacrh metainfo.xml at installation time so the only solution to be integrated into Log Search logic is to restart is Log Feeders manually in 2.5.x or 2.6.x Ambari versions. The new approach from 3.0 release, where new services will be detected by Log Search once installed and their logs indexed on the fly, will also work with 3rd party services installed trough stack extensions? Thanks again On Monday, October 23, 2017 11:52 PM, Miklos Gergely <[email protected]<mailto:[email protected]><mailto:[email protected]<mailto:[email protected]>>> wrote: If you check ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/metainfo.xml on branch-2.5 (https://github.com/apache/ambari/blob/branch-2.5/ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/metainfo.xml) there is indeed a <configuration-dependencies> element which enforces the restarting of HDP services upon the modification of *-logsearch-conf.xml, so there is nothing in the .js code. For additional 3rd party services nothing enforces it, and thus the Log Feeders should be restarted manually, without Ambari enforcing it. What may have confused you is that there are no such configuration-dependencies on trunk. This is because in the 3.0 release no restart will be needed for Log Search to support a new service, it will start to monitor their logs on the fly as they are installed. On 10/24/17, 1:59 AM, "Juanjo Marron" <[email protected]<mailto:[email protected]><mailto:[email protected]<mailto:[email protected]>>> wrote: >Hi all, > >I was following the details on this blog to add 3rd party services to Log >Search in Ambari 2.5.x >It has been really useful and the integration was pretty clean. > >We are just facing a minor issue forcing to restart Log Search service: > >How is the best way to implement the required restart in Log Search after >adding a new service to Log Search or after changing some of the >*-logsearch-conf.xml configurations for the new third party service in the UI? > >I can see it is working for HDP stack services. For instance changes in >zookeeper-logsearch-conf.xml requires Log Search restart in the UI. I dont >see <config-dependecies> in the metainfo.xml of log search is used to force >this behavior. >Maybe it is happening in the UI .js code, but additional 3rd party services >-installed after HDP- we shouldn't hack the .js code, right? > >Thanks for your answers
