TT1103 opened a new issue, #9415: URL: https://github.com/apache/pinot/issues/9415
We noticed high disk usage on the minion servers and it appears that the RealtimeToOfflineSegmentsTask sometimes leaves the segments on disk. We also noticed that Pinot is keeping a large amount deleted files open, which might be a memory management bug. Example `lsof -n | grep deleted` output: ``` java 106586 8183 TaskState username 634w REG 252,0 7401250816 5887067 /tmp/PinotMinion/data/RealtimeToOfflineSegmentsTask/tmp-6fe9e9c9-238c-4b99-8018-094f32bbad38/workingDir/mapper_output/0/record.offset (deleted) java 106586 8183 TaskState username 635w REG 252,0 128791597056 5887068 /tmp/PinotMinion/data/RealtimeToOfflineSegmentsTask/tmp-6fe9e9c9-238c-4b99-8018-094f32bbad38/workingDir/mapper_output/0/record.data (deleted) java 106586 8184 HelixTask username 634w REG 252,0 7401250816 5887067 /tmp/PinotMinion/data/RealtimeToOfflineSegmentsTask/tmp-6fe9e9c9-238c-4b99-8018-094f32bbad38/workingDir/mapper_output/0/record.offset (deleted) ``` see slack thread: https://apache-pinot.slack.com/archives/C011C9JHN7R/p1663342366477189 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
