GSharayu opened a new issue #6796: URL: https://github.com/apache/incubator-pinot/issues/6796
The refresh thread maintains a single global queue of realtime text index readers across all consuming segments across all tables. A reader is polled, refreshed and supposed to be added back to the queue so that it can be refreshed again in the next cycle. There is a bug in the refresh thread code as it somewhere fails to add the polled reader back to the queue. This will introduce the lag between documents being added to realtime text index and appearing in search results by the reader since the reader snapshot will be refreshed exactly once. The problem was recently reported in open source as the client was ingesting documents at time T, they appeared in search results at time T and T + X, but any documents added after T + X were not being reported in search results until long time when the consuming segment finally committed and converted to offline at which point obviously everything became available. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
