kkonstantine commented on pull request #8259: URL: https://github.com/apache/kafka/pull/8259#issuecomment-879532122
The idea that the `DelegatingClassLoader` did not have to be parallel capable originated to the fact that it doesn't load classes directly. It delegates loading either to the appropriate PluginClassLoader directly via composition, or to the parent by calling `super.loadClass`. The latter is the key point of why we need to make the `DelegatingClassLoader` also parallel capable even though it doesn't load a class. Because inheritance is used (via a call to `super.loadClass`) and not composition (via a hypothetical call to `parent.loadClass`, which is not possible because `parent` is a private member of the base abstract class `ClassLoader`) when `getClassLoadingLock` is called in `super.loadClass` it checks that actually the derived class (here an instance of `DelegatingClassLoader`) is not parallel capable and therefore ends up not applying fine-grain locking during classloading even though the parent clasloader is used actually load the classes. Based on the above, I added a last commit to mark the `DelegatingClassLoader` as parallel capable. I've tested both classloader types being parallel capable in a variety of scenarios with multiple connectors, SMTs and converters and a deadlock did not reproduce. Of course reproducing the issue is difficult without the specifics of the jar layout to begin with. The possibility of a deadlock is still not zero, but also probably not exacerbated compared to the current code. The plugin that depends on other plugins to be loaded while it's loading its classes is the connector type plugin only and there are no inter-connector dependencies (a connector requiring another connector's classes to be loaded while loading its own). With that in mind, a deadlock should be even less possible now. In the future we could consider introducing deadlock recovery methods to get out of this type of situation if necessary. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
