Ada Wong created FLINK-25804:
--------------------------------
Summary: Loading and running connector code use separated
ClassLoader.
Key: FLINK-25804
URL: https://issues.apache.org/jira/browse/FLINK-25804
Project: Flink
Issue Type: New Feature
Components: API / Core, Connectors / Common, Table SQL / Runtime
Affects Versions: 1.14.3
Reporter: Ada Wong
When we use multiple connectors could have class conflicts. This class conflict
can not be solved by shade.
The following is example code.
CREATE TABLE es6 (
user_id STRING,
user_name STRING,
PRIMARYKEY (user_id) NOT ENFORCED
) WITH (
'connector' = 'elasticsearch-6',
'hosts' = 'http://localhost:9200',
'index' = 'users',
'document-type' = 'foo'
);
CREATE TABLE es7 (
user_id STRING,
user_name STRING,
PRIMARYKEY (user_id) NOT ENFORCED
) WITH (
'connector' = 'elasticsearch-7',
'hosts' = 'http://localhost:9200',
'index' = 'users'
);
CREATE TABLE ods (
user_id STRING,
user_name STRING
) WITH (
'connector' = 'datagen'
);
INSERT INTO es6 SELECT user_id, user_name FROM ods;
INSERT INTO es7 SELECT user_id, user_name FROM ods;
{code:java}
CREATE TABLE es6 (
user_id STRING,
user_name STRING,
PRIMARYKEY (user_id) NOT ENFORCED
) WITH (
'connector' = 'elasticsearch-6',
'hosts' = 'http://localhost:9200',
'index' = 'users',
'document-type' = 'foo'
);
CREATE TABLE es7 (
user_id STRING,
user_name STRING,
PRIMARYKEY (user_id) NOT ENFORCED
) WITH (
'connector' = 'elasticsearch-7',
'hosts' = 'http://localhost:9200',
'index' = 'users'
);
CREATE TABLE ods (
user_id STRING,
user_name STRING
) WITH (
'connector' = 'datagen'
);
INSERT INTO es6 SELECT user_id, user_name FROM ods;
INSERT INTO es7 SELECT user_id, user_name FROM ods;{code}
Inspird by PulginManager, PluginFileSystemFactory and
ClassLoaderFixingFileSystem class.
Could we create many ClassLoaderFixing* class to avoid class conflict. Such as
ClassLoaderFixingDynamicTableFactory, ClassLoaderFixingSink or
ClassLoaderFixingSinkFunction.
If we use ClassLoader fixing, each call SinkFunction#invoke will switch
classloader by Thread#currentThread()#setContextClassLoader(). Does
setContextClassLoader() has heavy overhead of setContextClassLoader()?
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
