Great!!
Thanks one more time for all the details provided, Miklos
All the best,
Juanjo
On Tuesday, October 24, 2017 4:40 PM, Miklos Gergely
<[email protected]> wrote:
You are correct, in 3.0 the Log Feeder configuration is not editable as an
Ambari property. The json genereated by the file
input.config-<service_name>.json.j2 will bootstrap this configuration handled
by Log Search internally, after it was parsed by any of the Log Feeders for the
first time, later it will have no longer any effect if it was modified. This
configuration can be edited later only on the Log Search UI on a new window
specifically for editing configurations, which is not ready yet, or via calling
the REST API of the Log Search server directly. We’ll have some documentation
detailing how to handle configurations as soon as the UI is finished.
> The mapping among Ambari components and LogIds which used to be a property,
> is happening now based on the service metainfo.xml and the definition of
> LogIds in that file. is that correct?
Yes, that’s correct.
From: Juanjo Marron <[email protected]<mailto:[email protected]>>
Reply-To: Juanjo Marron
<[email protected]<mailto:[email protected]>>
Date: Tuesday, October 24, 2017 at 11:53 PM
To: "[email protected]<mailto:[email protected]>"
<[email protected]<mailto:[email protected]>>, default
<[email protected]<mailto:[email protected]>>
Subject: Re: 3rd party services requiring to restart Log Search
OK. I Understand now. Thanks for the info Miklos
So these templates input.config-<service_name>.json.j2 are just the content
property in the <service-name>-logsearch-conf.xml, right?
The use of templates seems similar to the old design of Log Search integration
before <service-name>-logsearch-conf.xml came into the game, but avoiding to
edit the HadoopServiceConfig.json.j2 file. That's great!
Are these templates still exposed in the Ambari UI as they were before as part
of the <service-name>-logsearch-conf.xml file?
I saw <service-name>-logsearch-conf.xml files are now removed from services
definition. So I assume user's cant not edit the templates directly in the UI.
If they are edited at file level, do they get copied over to Log Feeder config
directory. Hook will be just work after the install. The wont be updated if
there are later changes, right?
The mapping among Ambari components and LogIds which used to be a property, is
happening now based on the service metainfo.xml and the definition of LogIds in
that file. is that correct?
Sorry for all the questions but we are currently worting on the integration of
3rd party services into Log Search and I could not details in the JIRA or
documentation explaining the future changes.
Thanks again
On Tuesday, October 24, 2017 11:24 AM, Miklos Gergely
<[email protected]<mailto:[email protected]>> wrote:
Yes, if they have a file called input.config-<service_name>.json.j2 at
package/templates in their stack definition, and they are installed with
Ambari. There is a post install hook in Ambari looking for such files, and if
it is found in the stack definition it is copied into the Log Feeder config
directory, even if Log Feeder is not even installed yet. It is a jinja file
where you can reference configuration properties, e.g. log directory settings.
Check out Zookeeper’s configuration for example:
https://github.com/apache/ambari/blob/trunk/ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5/package/templates/input.config-zookeeper.json.j2
From: Juanjo Marron
<[email protected]<mailto:[email protected]><mailto:[email protected]<mailto:[email protected]>>>
Reply-To: Juanjo Marron
<[email protected]<mailto:[email protected]><mailto:[email protected]<mailto:[email protected]>>>
Date: Tuesday, October 24, 2017 at 7:18 PM
To: default
<[email protected]<mailto:[email protected]><mailto:[email protected]<mailto:[email protected]>>>,
"[email protected]<mailto:[email protected]><mailto:[email protected]<mailto:[email protected]>>"
<[email protected]<mailto:[email protected]><mailto:[email protected]<mailto:[email protected]>>>
Subject: Re: 3rd party services requiring to restart Log Search
Thanks Miklos for your quick response and the details provided.
True, I was taking a look just to trunk code. I see now all the
*-logsearch-conf.xml files listed as <configuration-dependencies> for the HDP
stack services in branch-2.5, which explains the required restart.
A 3rd party service won't be able to modify Log Seacrh metainfo.xml at
installation time so the only solution to be integrated into Log Search logic
is to restart is Log Feeders manually in 2.5.x or 2.6.x Ambari versions.
The new approach from 3.0 release, where new services will be detected by Log
Search once installed and their logs indexed on the fly, will also work with
3rd party services installed trough stack extensions?
Thanks again
On Monday, October 23, 2017 11:52 PM, Miklos Gergely
<[email protected]<mailto:[email protected]><mailto:[email protected]<mailto:[email protected]>>>
wrote:
If you check
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/metainfo.xml
on branch-2.5
(https://github.com/apache/ambari/blob/branch-2.5/ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/metainfo.xml)
there is indeed a <configuration-dependencies> element which enforces the
restarting of HDP services upon the modification of *-logsearch-conf.xml, so
there is nothing in the .js code. For additional 3rd party services nothing
enforces it, and thus the Log Feeders should be restarted manually, without
Ambari enforcing it.
What may have confused you is that there are no such configuration-dependencies
on trunk. This is because in the 3.0 release no restart will be needed for Log
Search to support a new service, it will start to monitor their logs on the fly
as they are installed.
On 10/24/17, 1:59 AM, "Juanjo Marron"
<[email protected]<mailto:[email protected]><mailto:[email protected]<mailto:[email protected]>>>
wrote:
>Hi all,
>
>I was following the details on this blog to add 3rd party services to Log
>Search in Ambari 2.5.x
>It has been really useful and the integration was pretty clean.
>
>We are just facing a minor issue forcing to restart Log Search service:
>
>How is the best way to implement the required restart in Log Search after
>adding a new service to Log Search or after changing some of the
>*-logsearch-conf.xml configurations for the new third party service in the UI?
>
>I can see it is working for HDP stack services. For instance changes in
>zookeeper-logsearch-conf.xml requires Log Search restart in the UI. I dont
>see <config-dependecies> in the metainfo.xml of log search is used to force
>this behavior.
>Maybe it is happening in the UI .js code, but additional 3rd party services
>-installed after HDP- we shouldn't hack the .js code, right?
>
>Thanks for your answers
