Thanks! Is there a way to get the latest sequence number of all shards of a kinesis stream?
On Mon, Nov 14, 2016 at 5:43 PM, Takeshi Yamamuro <linguin....@gmail.com> wrote: > Hi, > > The time interval can be controlled by `IdleTimeBetweenReadsInMillis` in > KinesisClientLibConfiguration > though, > it is not configurable in the current implementation. > > The detail can be found in; > https://github.com/apache/spark/blob/master/external/ > kinesis-asl/src/main/scala/org/apache/spark/streaming/ > kinesis/KinesisReceiver.scala#L152 > > // maropu > > > On Sun, Nov 13, 2016 at 12:08 AM, Shushant Arora < > shushantaror...@gmail.com> wrote: > >> *Hi * >> >> *is **spark.streaming.blockInterval* for kinesis input stream is >> hardcoded to 1 sec or is it configurable ? Time interval at which receiver >> fetched data from kinesis . >> >> Means stream batch interval cannot be less than >> *spark.streaming.blockInterval >> and this should be configrable , Also is there any minimum value for >> streaming batch interval ?* >> >> *Thanks* >> >> > > > -- > --- > Takeshi Yamamuro >
