clintropolis opened a new pull request, #18799: URL: https://github.com/apache/druid/pull/18799
### Description This PR fixes an issue with MSQ input file validation that results in creating a`TooManyInputFilesFault ` from the number of an individual worker instead of totals across all workers. This lead to incorrect error messaging, since the error message advises on the minimum number of workers required to run the work order, so by only considering a single workers inputs it would advise to have at least the number of workers required to run _that workers inputs_, even though the number of actual workers might far exceed that. For example, the adjusted test in this PR has 8 workers to run 100k inputs, before the changes in this PR the error message would be something like: ``` Too many input files/segments [14286] encountered. Maximum input files/segments per worker is set to [10000]. Try breaking your query up into smaller queries, or increasing the number of workers to at least [2] by... ``` The number of files is less than the 100k total for the job, and the "... increasing the number of workers to at least [2] ..." isn't helpful because we already have 8 workers. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
