Hi Arpit, 1. You can check the external view of the real-time table (via swagger api or zk browser in the console). Segments showing as ONLINE are flushed to disk and ones showing as CONSUMING are still in memory and not committed to disk yet. 2. Can you run the debug api from swagger to see if any errors in server?
Also, for faster turnaround, please join the Apache Pinot slack community as well. Thanks Mayank > On Oct 20, 2021, at 3:10 AM, Arpit Jain <jain.arp...@gmail.com> wrote: > > > Hi, > > I have setup a Pinot cluster 0.8.0 for real time data ingestion from Kafka. > It is able to consume data but it just consumes 100000 docs and stops I > believe. > Reading the docs, it should flush after a certain period of time/rows but I > think thats not happening. > I have below questions: > 1. How do I confirm if its flushing to disk? > 2. Why it is only consuming 100k docs ? > My settings are: > segment.flush.threshold.rows:"0" > segment.flush.threshold.size:"10000000" > segment.flush.threshold.time:"6h" > segment.flush.segment.size:"150M" > > Any inputs welcome. > > Regards, > Arpit --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@pinot.apache.org For additional commands, e-mail: users-h...@pinot.apache.org
