Re: [DISCUSS] Connectors, Formats, and even User Code should also be pluggable.

2023-08-10 Thread liu ron
Hi, zhiqiang Thanks for driving this proposal, it looks very interesting. As we all know, Flink has a very powerful connector ecosystem, and as Jing said, all the connectors are currently being externalized, and it would be very useful to design a pluggable mechanism for connectors, which could

Re: [DISCUSS] Connectors, Formats, and even User Code should also be pluggable.

2023-08-10 Thread Thor God
I'm interested in this, we often have linker dependency conflicts and it takes a lot of work to resolve dependency conflicts. Benchao Li 于2023年7月25日周二 20:51写道: > I agree with Jing that the current doc is quite preliminary, and needs to > be turned into a FLIP. > > I'll be very interested in

Re: [DISCUSS] Connectors, Formats, and even User Code should also be pluggable.

2023-07-25 Thread Benchao Li
I agree with Jing that the current doc is quite preliminary, and needs to be turned into a FLIP. I'll be very interested in this FLIP, and looking forward to it. We've suffered from reshading/relocating heavy connector dependencies for a long time. Besides, we've implemented a plugin mechanism

Re: [DISCUSS] Connectors, Formats, and even User Code should also be pluggable.

2023-07-20 Thread Jing Ge
Hi Zhiqiang, Thanks for your proposal. The idea is very interesting! Since almost all connectors are or will be externalized, the pluggable design you suggested could help reduce the development effort. As you mentioned, the attached doc contains only your preliminary idea. I would suggest that

[DISCUSS] Connectors, Formats, and even User Code should also be pluggable.

2023-07-14 Thread zhiqiang li
Hi devs, I have observed that in [1], connectors and formats are pluggable, allowing user code to be easily integrated. The advantages of having pluggable connectors are evident, as it helps avoid conflicts between different versions of jar packages. If classloader isolation is not used,