cbalci opened a new issue, #13149: URL: https://github.com/apache/pinot/issues/13149
We observed this interesting behavior from a Realtime table. For a single segment, something apparently went wrong at commit time that the server wasn't able to commit the segment to disk and remove files under 'consumers'. However the ingestion continued without an issue and the server has been consuming and sealing segments since then just fine. At this point this segment: - Shows 'Online' in ZK metadata (not 'Consuming') - Server has three preallocated files under `consumers/`, which are still memory mapped - No compressed (.tar.gz) segment file on disk - Some error logs from `RealtimeSegmentValidationManager` (controller) which attempted to upload segment files to deep store but failed. (Unfortunately segment build time logs from the server are lost due to retention) My two questions are: - How is it possible for a segment to get stuck in this limbo state where it finished consumption, but wasn't properly sealed. - Having the preallocated 'consumer' files, is there a practical way to rerun the commit routine and create the segment files after the fact. cc @chenboat @eaugene -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
