Hi Maxim,

in general, we have used shading in the past to avoid dependency hells for
users. We are in the process to replace more and more relocations with
plugins having a different classloader. However, I can't tell you if that
will also work for zookeeper and when that would happen.

If you manage to find out how Hadoop is solving that, I'd appreciate if you
could let us know.



On Fri, Feb 7, 2020 at 5:02 PM Maxim Parkachov <lazy.gop...@gmail.com>

> Hi Chesnay,
> I managed to re-compile with MapR zookeeper and can confirm that it works
> with HA as well.
> Still I find it strange that HA uses shadow version of zookeeper instead
> of version from classpath how it is done for hadoop.
> Thanks,
> Maxim.
> On Wed, Feb 5, 2020 at 3:43 PM Chesnay Schepler <ches...@apache.org>
> wrote:
>> No, since a) HA will never use classes from the user-jar and b) zookeeper
>> is relocated to a different package (to avoid conflicts) and hence any
>> replacement has to follow the same relocation convention.
>> On 05/02/2020 15:38, Maxim Parkachov wrote:
>> Hi Chesnay,
>> thanks for advise. Will it work if I include MapR specific zookeeper in
>> job dependencies and still use out-of-box Flink binary distribution ?
>> Regards,
>> Maxim.
>> On Wed, Feb 5, 2020 at 3:25 PM Chesnay Schepler <ches...@apache.org>
>> wrote:
>>> You must rebuild Flink while overriding zookeeper.version property to
>>> match your MapR setup.
>>> For example: mvn clean package -Dzookeeper.version=3.4.5-mapr-1604
>>> Note that you will also have to configure the MapR repository in your
>>> local setup as described here
>>> <https://maven.apache.org/guides/mini/guide-multiple-repositories.html>.
>>> On 05/02/2020 15:12, Maxim Parkachov wrote:
>>> Hi everyone,
>>> I have already written about issue with Flink 1.9 on secure MapR cluster
>>> and high availability. The issue was resolved with custom compiled Flink
>>> with vendor mapr repositories enabled. The history could be found
>>> https://www.mail-archive.com/user@flink.apache.org/msg28235.html
>>> Unfortunately, in current 1.10 RC vendor repositories were removed and
>>> I'm failing to get working configuration.  Current situation with 1.10 RC
>>> and secure MapR cluster:
>>> 1. Without HA, Flink uses class path provided zookeeper jar (mapr
>>> specific) and everything works fine.
>>> 2. With HA enabled, Flink uses shaded zookeeper  
>>> (org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ClientCnxn)
>>> which  doesn't have MapR specific changes and fails to authenticate.
>>> I would really appreciate any help in resolving this issue.I'm ready to
>>> provide any required details.
>>> Regards,
>>> Maxim.

Reply via email to