Re: Tracking / estimating job progress

2016-05-13 Thread Dood 

On 5/13/2016 10:39 AM, Anthony May wrote:
It looks like it might only be available via REST, 
http://spark.apache.org/docs/latest/monitoring.html#rest-api


Nice, thanks!



On Fri, 13 May 2016 at 11:24 Dood@ODDO > wrote:


On 5/13/2016 10:16 AM, Anthony May wrote:
>

http://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.SparkStatusTracker
>
> Might be useful

How do you use it? You cannot instantiate the class - is the
constructor
private? Thanks!

>
> On Fri, 13 May 2016 at 11:11 Ted Yu 
> >> wrote:
>
> Have you looked
> at
core/src/main/scala/org/apache/spark/ui/jobs/JobProgressListener.scala
> ?
>
> Cheers
>
> On Fri, May 13, 2016 at 10:05 AM, Dood@ODDO

> >>
wrote:
>
> I provide a RESTful API interface from scalatra for
launching
> Spark jobs - part of the functionality is tracking these
jobs.
> What API is available to track the progress of a particular
> spark application? How about estimating where in the
total job
> progress the job is?
>
> Thanks!
>
>
 -
> To unsubscribe, e-mail:
user-unsubscr...@spark.apache.org

> >
> For additional commands, e-mail:
user-h...@spark.apache.org 
> >
>
>


-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org

For additional commands, e-mail: user-h...@spark.apache.org





-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org

Re: Tracking / estimating job progress

2016-05-13 Thread Anthony May 
It looks like it might only be available via REST,
http://spark.apache.org/docs/latest/monitoring.html#rest-api

On Fri, 13 May 2016 at 11:24 Dood@ODDO  wrote:

> On 5/13/2016 10:16 AM, Anthony May wrote:
> >
> http://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.SparkStatusTracker
> >
> > Might be useful
>
> How do you use it? You cannot instantiate the class - is the constructor
> private? Thanks!
>
> >
> > On Fri, 13 May 2016 at 11:11 Ted Yu  > > wrote:
> >
> > Have you looked
> > at
> core/src/main/scala/org/apache/spark/ui/jobs/JobProgressListener.scala
> > ?
> >
> > Cheers
> >
> > On Fri, May 13, 2016 at 10:05 AM, Dood@ODDO  > > wrote:
> >
> > I provide a RESTful API interface from scalatra for launching
> > Spark jobs - part of the functionality is tracking these jobs.
> > What API is available to track the progress of a particular
> > spark application? How about estimating where in the total job
> > progress the job is?
> >
> > Thanks!
> >
> >
>  -
> > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
> > 
> > For additional commands, e-mail: user-h...@spark.apache.org
> > 
> >
> >
>
>
> -
> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
> For additional commands, e-mail: user-h...@spark.apache.org
>
>

Re: Tracking / estimating job progress

2016-05-13 Thread Dood 

On 5/13/2016 10:16 AM, Anthony May wrote:

http://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.SparkStatusTracker

Might be useful


How do you use it? You cannot instantiate the class - is the constructor 
private? Thanks!




On Fri, 13 May 2016 at 11:11 Ted Yu > wrote:


Have you looked
at core/src/main/scala/org/apache/spark/ui/jobs/JobProgressListener.scala
?

Cheers

On Fri, May 13, 2016 at 10:05 AM, Dood@ODDO > wrote:

I provide a RESTful API interface from scalatra for launching
Spark jobs - part of the functionality is tracking these jobs.
What API is available to track the progress of a particular
spark application? How about estimating where in the total job
progress the job is?

Thanks!

-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org

For additional commands, e-mail: user-h...@spark.apache.org






-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org

Re: Tracking / estimating job progress

2016-05-13 Thread Anthony May 
http://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.SparkStatusTracker

Might be useful

On Fri, 13 May 2016 at 11:11 Ted Yu  wrote:

> Have you looked
> at core/src/main/scala/org/apache/spark/ui/jobs/JobProgressListener.scala ?
>
> Cheers
>
> On Fri, May 13, 2016 at 10:05 AM, Dood@ODDO  wrote:
>
>> I provide a RESTful API interface from scalatra for launching Spark jobs
>> - part of the functionality is tracking these jobs. What API is available
>> to track the progress of a particular spark application? How about
>> estimating where in the total job progress the job is?
>>
>> Thanks!
>>
>> -
>> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
>> For additional commands, e-mail: user-h...@spark.apache.org
>>
>>
>

Re: Tracking / estimating job progress

2016-05-13 Thread Ted Yu 
Have you looked
at core/src/main/scala/org/apache/spark/ui/jobs/JobProgressListener.scala ?

Cheers

On Fri, May 13, 2016 at 10:05 AM, Dood@ODDO  wrote:

> I provide a RESTful API interface from scalatra for launching Spark jobs -
> part of the functionality is tracking these jobs. What API is available to
> track the progress of a particular spark application? How about estimating
> where in the total job progress the job is?
>
> Thanks!
>
> -
> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
> For additional commands, e-mail: user-h...@spark.apache.org
>
>

Tracking / estimating job progress

2016-05-13 Thread Dood 
I provide a RESTful API interface from scalatra for launching Spark jobs 
- part of the functionality is tracking these jobs. What API is 
available to track the progress of a particular spark application? How 
about estimating where in the total job progress the job is?


Thanks!

-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org