Hi Max, Thanks for the help. I will certainly look into shading the library, but my setup is very straightforward if someone is interested in reproducing this issue.
I started and AWS EMR cluster version 5.30 with flink 1.10.0 installed. I created the beam wordcount example project using maven (beam version 2.22). Finally, I started a flink yarn session and ran the example project connected to the flink master. If it is easier to see exact code examples to re-create from scratch I would be happy to provide them. Thanks, Jesse On 6/24/20, 4:01 AM, "Maximilian Michels" <[email protected]> wrote: Hi Jesse, This is hard to debug without knowing more about the setup. You have conflicting versions of the Jackson library. One is present in Beam, one may be loaded by the AWS setup. You mentioned in the issue that using parent-child first classloading did not resolve the issue. I'd suggest to relocate (shade) Jackson in the Jar you submit to AWS. -Max On 23.06.20 17:22, Jesse Lord wrote: > Hi Max, > > The error message shows up in the flink web UI, so I think it must be reaching the cluster. > > For the portable runner the error is listed in the docker container output as well but I assume that is just receiving the error message from the flink cluster. > > Thanks, > Jesse > > On 6/23/20, 8:11 AM, "Maximilian Michels" <[email protected]> wrote: > > Hey Jesse, > > Could you share the context of the error? Where does it occur? In the > client code or on the cluster? > > Cheers, > Max > > On 22.06.20 18:01, Jesse Lord wrote: > > I am trying to run the wordcount quickstart example on a flink cluster > > on AWS EMR. Beam version 2.22, Flink 1.10. > > > > > > > > I get the following error: > > > > > > > > ERROR:root:java.util.ServiceConfigurationError: > > com.fasterxml.jackson.databind.Module: Provider > > com.fasterxml.jackson.module.jaxb.JaxbAnnotationModule not a subtype > > > > > > > > This happens with both the portable runner (using python SDK) and the > > classic flink runner using the quickstart maven project. > > > > > > > > I think this error relates to this issue: > > https://nam05.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FBEAM-9239&data=02%7C01%7Cjlord%40vectra.ai%7C516cc4f79f0e46cf966008d8182dee7f%7Ca6cc66bcf41945c2a9c28ff4ab685f2d%7C1%7C0%7C637285932846323904&sdata=HwkC7phfykOsJ8kS0Vz6STc9LbaCFeHmQPHZXbp1zng%3D&reserved=0. Based on the comments > > from this issue I tried adjusting parameters for whether flink > > prioritizes loading child (user) jars or parent (flink) jars in the > > classpath but it did not resolve the issue. > > > > > > > > Looking for any suggestions that might help as a workaround and > > wondering if I should open a new jira issue or only add my comment to > > the existing issue (which I have already done). > > > > > > > > Thanks, > > > > Jesse > > >
