[
https://issues.apache.org/jira/browse/FLINK-15786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17046423#comment-17046423
]
Guowei Ma commented on FLINK-15786:
-----------------------------------
[~fly_in_gis] Yes. But, there are some differences between FileSystem and
Connector such as
# The DataStream user could new a connector object directly
# The Source/Sink operator would be deserialized at TM sided.
# The Source operator might construct a user class provided by the user at the
runtime.
In general, these differences would lead to change the
FlinkUsercodeClassloader. For example, FlinkUsercodeClassLoader might need some
extra connector classloaders, which could be used to find the connector and its
depend on class.
[~pnowojski] I might miss some scenarios, could you give me some of your
concerns? Thanks you !
> Load connector code with separate classloader
> ---------------------------------------------
>
> Key: FLINK-15786
> URL: https://issues.apache.org/jira/browse/FLINK-15786
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Task
> Reporter: Guowei Ma
> Priority: Major
> Labels: usability
>
> Currently, connector code can be seen as part of user code. Usually, users
> only need to add the corresponding connector as a dependency and package it
> in the user jar. This is convenient enough.
> However, connectors usually need to interact with external systems and often
> introduce heavy dependencies, there is a high possibility of a class conflict
> of different connectors or the user code of the same job. For example, every
> one or two weeks, we will receive issue reports relevant with connector class
> conflict from our users. The problem can get worse when users want to analyze
> data from different sources and write output to different sinks.
> Using separate classloader to load the different connector code could resolve
> the problem.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
