Hi, I am new to NiFi and I am reading off a Kafka topic that has 3 partitions. In my Nifi flow, I have 3 ConsumeKafka processors with the same groupId and Topic. However, when I push large datasets (e.g., 200,000+), 300-400 records don't make it to the next processor. This only happens when I have the Concurrent Tasks in the Scheduling Tab set to more than 1 (e.g., 2 or 3). If I have the Concurrent Tasks set to 1 then all the records make it through to the next processor just fine.
I may need to define kafka.partitions to have each Nifi processor point to a specific Kafka partition, but am not sure where/how to do that. I tried adding it to the properties, but that doesn't work. Has anyone else worked through this issue? I am using Nifi 1.1.1 and Kafka 0.9 -- Sam
