Hi, We have noticed recently that no matter what batch size we use for Metron indexing feeds, as long as we start using different batch size for different Metron feeds, indexing topology throughput will start dropping due to the high error rate! So I was wondering whether based on the current indexing topology design, we have to choose the same batch size for all the feeds or not. Otherwise, throughout will be dropped. I assume since it is acceptable to use different batch sizes for different feeds, it is not expected by design.
Moreover, I have noticed in practice that even if we change the batch size, it will not affect the messages that are already in enrichments or indexing topics, and it will only affect the new messages that are coming to the parser. Therefore, we need to let all the messages pass the indexing topology so that we can change the batch size! It would be great if we can have more details regarding the design of this section so we can understand our observations are based on the design or some kind of bug. Regards, Ali
