Hi, I have setup a Pinot cluster 0.8.0 for real time data ingestion from Kafka. It is able to consume data but it just consumes 100000 docs and stops I believe. Reading the docs, it should flush after a certain period of time/rows but I think thats not happening. I have below questions: 1. How do I confirm if its flushing to disk? 2. Why it is only consuming 100k docs ? My settings are: segment.flush.threshold.rows:"0" segment.flush.threshold.size:"10000000" segment.flush.threshold.time:"6h" segment.flush.segment.size:"150M"
Any inputs welcome. Regards, Arpit
