jihoonson commented on issue #11165: URL: https://github.com/apache/druid/issues/11165#issuecomment-840119258
@kfaraz thanks for adding more details. Please find my new comments below. > > In DruidTypedException, what does "typed" mean? > > The `Type` refers to the fact that this exception is raised for a specific `ErrorType` (contains code, moduleName and message format). Sorry, I couldn't think of a more appropriate name for this. 1. No worries. I think it should be something else though as now we are going to focus on the ingestion side. Maybe `IngestionException`? 2. FYI, `TaskQueue.attachCallbacks()` is used only in the local mode of overlord (see `druid.indexer.runner.type` in https://druid.apache.org/docs/latest/configuration/index.html#overlord-operations). The local mode is not used in production, but we should handle it as you suggested. 3. In peons and indexers, tasks can currently throw _any exception anywhere_. Peons and indexers catch those exceptions and report task failures to the overlord. I think you will want to change this to not allowing tasks to throw any exceptions, but only the new type of exceptions (`DruidTypedException` or `IngestionException`) because you should be able to convert them to `TaskStatus.failure()`. This means you will have to modify all places where the task throws any exception to use the new exception type. I'm not sure if there is a less invasive way than this.. 4. What do you think about exposing error codes to users? We cannot avoid it in some cases anyway such as where the extension is unloaded and the overlord doesn't understand the error code of existing task statuses. It seems more consistent to me but not harm. Maybe `TaskStatus` can only show the error code and `TaskReport` can show the full details of the error. 5. Semi-related, have you thought about adding the new `CANCELED` state for ingestion tasks? I think you may want to do it because the error code doesn't seem reasoable for canceled tasks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
