jon-wei opened a new pull request #8107: Add CliIndexer process type and initial task runner implementation URL: https://github.com/apache/incubator-druid/pull/8107 First part of implementation for #7900 ### Description This PR adds the initial foundation for the work in #7900: - A new Indexer process type is added, running on port 8091 like MiddleManagers. - This process type is like MM but uses a new ThreadingTaskRunner, which runs Tasks using a thread pool instead of forking processes - A new interface, `AppenderatorsManager`, which creates and manages Appenderators for tasks, has been added. The AppenderatorsManager for the new Indexer process uses a shared query execution pool and per-datasource segment walkers/imelines for the Appenderators it creates. - The TaskReportWriter has been adjusted to support managing multiple tasks - The new Indexer process uses two ports, its configured port and (configured port + 1). The second port is used for ChatHandler requests, to give them a dedicated pool separate from the query processing connection pool. - Lookups are loaded once for the entire Indexer process No additional memory management for the tasks is present in this PR, they will use whatever has been configured in their task specs with no additional external control. This PR has: - [x] been self-reviewed. - [x] using the [concurrency checklist](https://github.com/apache/incubator-druid/blob/master/dev/code-review/concurrency.md) - [x] added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader. - [x] added unit tests or modified existing tests to cover new code paths. - [x] been tested in a test Druid cluster.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
