Unable to use "Batch Start Time" on worker nodes.

[Abhishek Anand]

Hi ,

I need to use batch start time in my spark streaming job.

I need the value of batch start time inside one of the functions that is
called within a flatmap function in java.

Please suggest me how this can be done.

I tried to use the StreamingListener class and set the value of a variable
inside the onBatchSubmitted function something like this :

public void onBatchSubmitted(StreamingListenerBatchSubmitted
batchSubmitted) {     batchstarttime =
batchSubmitted.batchInfo().batchTime().milliseconds();
  CommandLineArguments.BATCH_START_TIME = batchstarttime;
 }


But, the issue is that the BATCH_START_TIME set only when the batch starts.
I see in the worker logs that BATCH_START_TIME takes the default value and
is not set.


Please suggest how this can be achieved.



BR,
Abhi