Re: Spark Monitoring using Jolokia

[Gourav Sengupta]

Hi,

I am totally confused here, may be because I do not exactly understand
this, but why is this required? I have always used SPARK UI and found that
more than sufficient. And if you know a bit about how SPARK session works
then your performance does have a certain degree of predictability as well.

Sorry about this, but the input may really help.

I may be wrong, but I find such pursuits just a sheer exercise in creating
products which in reality are not needed.


Regards,
Gourav Sengupta

On Mon, Jan 8, 2018 at 1:55 PM, Irtiza Ali  wrote:

> Hello everyone,
>
> I am building a monitoring tool for the spark, for that I needs sparks
> metrics. I am using jolokia to get the metrics.
>
> I have a question that:
>
> Can I get all the metrics provided by the spark rest api using the Jolokia?
>
> How the spark rest api get the metrics internally?
>
>
> Thanks
>

Re: Spark Monitoring using Jolokia

[Thakrar, Jayesh]

And here's some more info on Spark Metrics

https://www.slideshare.net/JayeshThakrar/apache-bigdata2017sparkprofiling


From: Maximiliano Felice <maximilianofel...@gmail.com>
Date: Monday, January 8, 2018 at 8:14 AM
To: Irtiza Ali <i...@an10.io>
Cc: <user@spark.apache.org>
Subject: Re: Spark Monitoring using Jolokia

Hi!

I don't know very much about them, but I'm currently working in posting custom 
metrics into Graphite. I found useful the internals described in this library: 
https://github.com/groupon/spark-metrics<https://github.com/groupon/spark-metrics>

Hope this at least can give you a hint.

Best of luck!

2018-01-08 10:55 GMT-03:00 Irtiza Ali <i...@an10.io<mailto:i...@an10.io>>:
Hello everyone,

I am building a monitoring tool for the spark, for that I needs sparks metrics. 
I am using jolokia to get the metrics.

I have a question that:

Can I get all the metrics provided by the spark rest api using the Jolokia?

How the spark rest api get the metrics internally?


Thanks

Re: Spark Monitoring using Jolokia

[Maximiliano Felice]

Hi!

I don't know very much about them, but I'm currently working in posting
custom metrics into Graphite. I found useful the internals described in
this library: https://github.com/groupon/spark-metrics


Hope this at least can give you a hint.

Best of luck!

2018-01-08 10:55 GMT-03:00 Irtiza Ali :

> Hello everyone,
>
> I am building a monitoring tool for the spark, for that I needs sparks
> metrics. I am using jolokia to get the metrics.
>
> I have a question that:
>
> Can I get all the metrics provided by the spark rest api using the Jolokia?
>
> How the spark rest api get the metrics internally?
>
>
> Thanks
>

Spark Monitoring using Jolokia

[Irtiza Ali]

Hello everyone,

I am building a monitoring tool for the spark, for that I needs sparks
metrics. I am using jolokia to get the metrics.

I have a question that:

Can I get all the metrics provided by the spark rest api using the Jolokia?

How the spark rest api get the metrics internally?


Thanks