baisui1981 edited a comment on pull request #17521: URL: https://github.com/apache/flink/pull/17521#issuecomment-962551137
thanks @AHeise for your reply, Sorry, I have not explained a detail before。For better illustration, I drew a diagram as below: [http://qlangtech.oss-us-west-1.aliyuncs.com/Flink%20ClassLoader.png](http://qlangtech.oss-us-west-1.aliyuncs.com/Flink%20ClassLoader.png)  There is a prerequisite for the processing flow. We need to build a plug-in repository based on the http protocol. The user needs to deploy the plugin to the warehouse in advance there are four steps in the process: 1. build two classloader which is respect to the plugin bundle name ‘ kafka’ and ‘ kinesis’ . specially the jars relevant to the plugin is store in the remote plugin-warehouse. the gava with different version as you have mentioned will be loaded separately in different URLClassloader，So there will be no class conflict 2. compile the client side code and make a package which is named as 'user.jar' and submit to `Flink Cluster`. the 'user.jar' is compose of two component, one is classes and another is plugin meta such as 'kafka' and 'kinesis' is stored in the `jar manifest` Regarding a question you raised: >What I fail to understand is how the incompatible version of Guava are put into the user.jar. Actually，`Guava` artifact is not attached in 'user.jar' whereas it is store in the remote plugin warehouse in advance 3. when flink server side received client apply , create instance of `TISClientFirstClassLoader` which is extended from `FlinkUserCodeClassLoader`, and this instance is response for loading class which is relevant to the plugin bundle of 'kafka' and 'kinesis'. In the `TISClientFirstClassLoader` will build a classloader array from 'kafka' and 'kinesis' which is parsed from `user.jar` manifest, code as below: ``` java public TISClientFirstClassLoader(String[] plugins){ this.tisClassLoaders = new URLClassLoader[plugins.length]; int i = 0; for(String plugin : plugins){ this.tisClassLoaders[i++] = createChildClassLoader(plugin); } } private URLClassLoader createChildClassLoader(String plugin){ // get the urls which is belong to the plugin URL[] urls = ....; return new URLClassLoader(urls); } ``` 4. TISClientFirstClassLoader is response for loading class which is defined at client side, by the classloader array which have just created in the `3` step. code as below: ```java public Class<?> findClass(String name) throws ClassNotFoundException { for (ClassLoader loader : tisClassLoaders) { try { return loader.loadClass(name); } catch (ClassNotFoundException e) { // not found. try next } } return null; } ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
