Mayur, I'm interested on it as well. Can you send me?
Cheers, Otávio Carvalho. Undergrad. Student at Federal University of Rio Grande do Sul Porto Alegre, Brazil. 2014-05-23 11:00 GMT-03:00 Pierre Borckmans < pierre.borckm...@realimpactanalytics.com>: > That would be great, Mayur, thanks! > > Anyhow, to be more specific, my question really was the following: > > Is there any way to link events in the SparkListener to an action > triggered in your code? > > Cheers > > > > > *Pierre Borckmans* > Software team > > *Real**Impact* Analytics *| *Brussels Office > www.realimpactanalytics.com *| > *pierre.borckm...@realimpactanalytics.com<thierry.lib...@realimpactanalytics.com> > > *FR *+32 485 91 87 31 *| **Skype* pierre.borckmans > > > > > > > On 23 May 2014, at 10:17, Mayur Rustagi <mayur.rust...@gmail.com> wrote: > > We have an internal patched version of Spark webUI which exports > application related data as Json. We use monitoring systems as well as > alternate UI for that json data for our specific application. Found it much > cleaner. Can provide 0.9.1 version. > Would submit as a pull request soon. > > > Mayur Rustagi > Ph: +1 (760) 203 3257 > http://www.sigmoidanalytics.com > @mayur_rustagi <https://twitter.com/mayur_rustagi> > > > > On Fri, May 23, 2014 at 10:57 AM, Chester <chesterxgc...@yahoo.com> wrote: > >> This is something we are interested as well. We are planning to >> investigate more on this. If someone has suggestions, we would love to hear. >> >> Chester >> >> Sent from my iPad >> >> On May 22, 2014, at 8:02 AM, Pierre B < >> pierre.borckm...@realimpactanalytics.com> wrote: >> >> Hi Andy! >> >> Yes Spark UI provides a lot of interesting informations for debugging >> purposes. >> >> Here I’m trying to integrate a simple progress monitoring in my app ui. >> >> I’m typically running a few “jobs” (or rather actions), and I’d like to >> be able to display the progress of each of those in my ui. >> >> I don’t really see how i could do that using SparkListener for the moment >> … >> >> Thanks for your help! >> >> Cheers! >> >> >> >> >> *Pierre Borckmans* >> Software team >> >> *Real**Impact* Analytics *| *Brussels Office >> <http://www.realimpactanalytics.com/>www.realimpactanalytics.com *| *[hidden >> email] <http://user/SendEmail.jtp?type=node&node=6259&i=0> >> >> *FR *+32 485 91 87 31 *| **Skype* pierre.borckmans >> >> >> >> >> >> >> On 22 May 2014, at 16:58, andy petrella [via Apache Spark User List] <[hidden >> email] <http://user/SendEmail.jtp?type=node&node=6259&i=1>> wrote: >> >> SparkListener offers good stuffs. >> But I also completed it with another metrics stuffs on my own that use >> Akka to aggregate metrics from anywhere I'd like to collect them (without >> any deps on ganglia yet on Codahale). >> However, this was useful to gather some custom metrics (from within the >> tasks then) not really to collect overall monitoring information about the >> spark thingies themselves. >> For that Spark UI offers already a pretty good insight no? >> >> Cheers, >> >> aℕdy ℙetrella >> about.me/noootsab >> [image: aℕdy ℙetrella on about.me] >> >> <http://about.me/noootsab> >> >> >> On Thu, May 22, 2014 at 4:51 PM, Pierre B <<a href=" >> x-msg://7/user/SendEmail.jtp?type=node&node=6258&i=0" >> target="_top" rel="nofollow" link="external">[hidden email]> wrote: >> >>> Is there a simple way to monitor the overall progress of an action using >>> SparkListener or anything else? >>> >>> I see that one can name an RDD... Could that be used to determine which >>> action triggered a stage, ... ? >>> >>> >>> Thanks >>> >>> Pierre >>> >>> >>> >>> -- >>> View this message in context: >>> <http://apache-spark-user-list.1001560.n3.nabble.com/Use-SparkListener-to-get-overall-progress-of-an-action-tp6256.html> >>> http://apache-spark-user-list.1001560.n3.nabble.com/Use-SparkListener-to-get-overall-progress-of-an-action-tp6256.html >>> Sent from the Apache Spark User List mailing list archive at >>> <http://nabble.com/>Nabble.com <http://nabble.com/>. >>> >> >> >> >> ------------------------------ >> If you reply to this email, your message will be added to the >> discussion below: >> <http://apache-spark-user-list.1001560.n3.nabble.com/Use-SparkListener-to-get-overall-progress-of-an-action-tp6256p6258.html> >> http://apache-spark-user-list.1001560.n3.nabble.com/Use-SparkListener-to-get-overall-progress-of-an-action-tp6256p6258.html >> To unsubscribe from Use SparkListener to get overall progress of an >> action, click here. >> NAML<http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> >> >> >> >> ------------------------------ >> View this message in context: Re: Use SparkListener to get overall >> progress of an >> action<http://apache-spark-user-list.1001560.n3.nabble.com/Use-SparkListener-to-get-overall-progress-of-an-action-tp6256p6259.html> >> Sent from the Apache Spark User List mailing list >> archive<http://apache-spark-user-list.1001560.n3.nabble.com/>at >> Nabble.com <http://nabble.com/>. >> >> > >
