ankitsultana opened a new issue, #14838:
URL: https://github.com/apache/pinot/issues/14838
We are seeing segments running into ERROR state due to a race condition
between Segment Refresh message and Offline to Online transition. This
ultimately leads to `Inconsistent data read. Index data file xyz/columns.psf is
possibly corrupted`.
I haven't taken a deeper look into this, but I can share that the following
events happened around the same time (in no particular order):
- There was a node replacement.
- There was a segment compaction job that processes this segment around the
same time.
```
{"@timestamp":"2025-01-18T06:33:47.092+00:00","message":"Creating new
inverted index for segment: some_table__13__2492__20250114T2318Z, column:
lorem_ipsum","logger_name":"org.apache.pinot.segment.local.segment.index.loader.invertedindex.InvertedIndexHandler","thread_name":"HelixTaskExecutor-message_handle_thread_33","level":"INFO"}
{"@timestamp":"2025-01-18T06:33:47.098+00:00","message":"Need to create new
inverted index for segment: some_table__13__2492__20250114T2318Z, column:
foobar","logger_name":"org.apache.pinot.segment.local.segment.index.loader.invertedindex.InvertedIndexHandler","thread_name":"HelixTaskExecutor-message_handle_thread_43","level":"INFO"}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
