Re: Getting the batch time of the active batches in spark streaming

[Todd Nist]

Hi Abhi,

You should be able to register a
org.apache.spark.streaming.scheduler.StreamListener.

There is an example here that may help:
https://gist.github.com/akhld/b10dc491aad1a2007183 and the spark api docs
here,
http://spark.apache.org/docs/latest/api/java/org/apache/spark/scheduler/SparkListener.html
.

HTH,
-Todd

On Tue, Nov 24, 2015 at 4:50 PM, Abhishek Anand 
wrote:

> Hi ,
>
> I need to get the batch time of the active batches which appears on the UI
> of spark streaming tab,
>
> How can this be achieved in Java ?
>
> BR,
> Abhi
>

Re: Getting the batch time of the active batches in spark streaming

[Todd Nist]

Hi Abhi,

Sorry that was the wrong link should have been the StreamListener,
http://spark.apache.org/docs/latest/api/java/org/apache/spark/streaming/scheduler/StreamingListener.html

The BatchInfo can be obtained from the event, for example:

public void onBatchSubmitted(StreamingListenerBatchSubmitted batchSubmitted)
{ system.out.println("Start time: " +
batchSubmitted.batchInfo.processingStartTime)
}

Sorry for the confusion.

-Todd

On Tue, Nov 24, 2015 at 7:51 PM, Todd Nist  wrote:

> Hi Abhi,
>
> You should be able to register a
> org.apache.spark.streaming.scheduler.StreamListener.
>
> There is an example here that may help:
> https://gist.github.com/akhld/b10dc491aad1a2007183 and the spark api docs
> here,
> http://spark.apache.org/docs/latest/api/java/org/apache/spark/scheduler/SparkListener.html
> .
>
> HTH,
> -Todd
>
> On Tue, Nov 24, 2015 at 4:50 PM, Abhishek Anand 
> wrote:
>
>> Hi ,
>>
>> I need to get the batch time of the active batches which appears on the
>> UI of spark streaming tab,
>>
>> How can this be achieved in Java ?
>>
>> BR,
>> Abhi
>>
>
>