Christian, We happened to run into this exact situation a couple days ago while tuning the indexing topology. In our case we were testing HDFS write performance and ES wasn't involved at all. We eventually tracked it down to a bad topic and we suspect the offsets were messed up somehow. Creating a new topic resolved the problem. Not sure what the root cause is at the moment but hopefully this workaround can get you past it. Hope this helps.
Ryan On Wed, May 10, 2017 at 8:56 AM, Christian Tramnitz <[email protected]> wrote: > Is anyone aware of a config setting that could cause the indexing topology > to stop after writing 10k messages? > > > > I have a parser running, writing into the indexing topic. Upon restart, > the indexing topology (just ES enabled for now) picks up the latest (not > oldest non-committed!) messages and puts 10000 into the current index. Then > it silently stops. No logs (ES, Kafka, Storm) give an indication what’s > preventing it from going on. When restarting the topology, it does another > round of 10k messages and stops again. Very weird. > > > > This is on a cluster build from Friday’s master and bare metal > installation according to the documentation. > > > > > > Thanks, > > Christian >
