If you're using the direct stream, and don't have speculative execution turned on, there is one executor consumer created per partition, plus a driver consumer for getting the latest offsets. If you have fewer executors than partitions, not all of those consumers will be running at the same time.
The direct stream doesn't use consumer groups in the same way the kafka high level consumer does, but you should be able to pass group id in the kafka parameters. On Tue, Jun 21, 2016 at 9:56 AM, Guillermo Ortiz <konstt2...@gmail.com> wrote: > I use Spark Streaming with Kafka and I'd like to know how many consumers are > generated. I guess that as many as partitions in Kafka but I'm not sure. > Is there a way to know the name of the groupId generated in Spark to Kafka? --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org