Lian
LogSearch uses logfeeder reads the logs from the log file and pushes it to Ambari Infra Solr. If you know where your Sparks logs are configured to go, then you should be able to configure LogFeeder to send the logs to Ambari Infra Solr. Bosco From: Lian Jiang <[email protected]> Reply-To: <[email protected]> Date: Thursday, February 22, 2018 at 2:20 PM To: <[email protected]> Subject: Re: can log search support spark application logs? Thanks. I want a log search UI like splunk/solr/kabana. It is interesting to see how to feed my application logs to log search. On Thu, Feb 22, 2018 at 2:02 PM, Gonzalo Herreros <[email protected]> wrote: The logs are available through the Spark UI (which is the Application Master link in YARN UI) on the Executors tab (stderr and stdout) AFAIK, the log search can only index local files but you could have a cronjob downloading then directly from hdfs. Gonzalo On 22 February 2018 at 21:27, Lian Jiang <[email protected]> wrote: Thanks for any help. What's the best way to get spark application logs? yarn command line is not ideal for spark streaming applications. On Wed, Feb 21, 2018 at 10:41 AM, Lian Jiang <[email protected]> wrote: Can I make my spark application log available for ambari log search so that I can search them on log search UI? Or I have to setup another solr/elastic search cluster for my application logs? Thanks for any hints.
