gortiz opened a new issue, #9113: URL: https://github.com/apache/pinot/issues/9113
SegmentGenerationJobRunner.submitSegmentGenTask submits tasks to an executor. Each task has a try-catch that logs the problem and stops the execution in case a subclass of Exception is thrown. But this code does not log the problem if an Error is thrown. For example, if the JVM that executes SegmentGenerationJobRunner does not have enough heap memory to apply the generation, then Java will throw an OutOfMemoryError that will not be catched by SegmentGenerationJobRunner.submitSegmentGenTask. As a result, no log is printed to the user and the JVM continues to work. Eventually there is enough memory to sucessfully create the other segments and the process finish with a correct status but without generating some segments. This behavior is very misleading, as the user thinks that his data is correctly stored in Pinot, when in fact some segments have been ignored. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
