Hi,
@chris @tdas Referring to the latest integration documentation, it states the following: "A single Kinesis input DStream can read from multiple shards of a Kinesis stream by creating multiple KinesisRecordProcessor threads." But looking at the API and the example, each time we call KinesisUtils.createStream(), it is creating 1 dstream per shard and we should call it up to N shards per stream. But according to documentation, we can increase the number of threads so 1 dstream can read form multiple shards? Looking at the receiver code, there is only 1 thread and is not configurable? https://github.com/apache/spark/blob/master/extras/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisReceiver.scala#L170 Thanks, Liming
