This looks like a typical issue with classloading. kinesis is probably residing in flink-dist/lib while woodstock is added in your job.jar (or vice versa).
Could you try to use both jars in the same way? Alternatively, could you provide more information regarding your dependencies? On Tue, Apr 21, 2020 at 11:21 AM Fu, Kai <k...@amazon.com> wrote: > Hi, I’m using Flink 1.8 with JDK 8. > > > > *-- Best wishes* > > *Fu Kai* > > > > > > *From: *Chesnay Schepler <ches...@apache.org> > *Date: *Tuesday, April 21, 2020 at 5:15 PM > *To: *"Fu, Kai" <k...@amazon.com>, "user@flink.apache.org" < > user@flink.apache.org> > *Subject: *RE: [EXTERNAL] Unable to unmarshall response > (com.ctc.wstx.stax.WstxInputFactory cannot be cast to > javax.xml.stream.XMLInputFactory) > > > > Which Flink version are you using? > > > > On 21/04/2020 11:11, Fu, Kai wrote: > > Hi, > > > > I’m running Flink application on AWS Kinesis Flink platform to read a > kinesis stream from another account with assumed role, while I’m getting > exception like below. But it works when I’m running the application > locally, I’ve given all the related roles admin permission. Could anyone > help what’s the potential problem? > > > > [ > > "org.apache.flink.kinesis.shaded.com.amazonaws.SdkClientException: > Unable to unmarshall response (com.ctc.wstx.stax.WstxInputFactory cannot be > cast to javax.xml.stream.XMLInputFactory). Response Code: 200, Response > Text: OK", > > "\tat > org.apache.flink.kinesis.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleResponse(AmazonHttpClient.java:1738)", > > "\tat > org.apache.flink.kinesis.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleSuccessResponse(AmazonHttpClient.java:1434)", > > "\tat > org.apache.flink.kinesis.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1356)", > > "\tat > org.apache.flink.kinesis.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1139)", > > "\tat > org.apache.flink.kinesis.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:796)", > > "\tat > org.apache.flink.kinesis.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:764)", > > "\tat > org.apache.flink.kinesis.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:738)", > > "\tat > org.apache.flink.kinesis.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:698)", > > "\tat > org.apache.flink.kinesis.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:680)", > > "\tat > org.apache.flink.kinesis.shaded.com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:544)", > > "\tat > org.apache.flink.kinesis.shaded.com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:524)", > > "\tat > org.apache.flink.kinesis.shaded.com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.doInvoke(AWSSecurityTokenServiceClient.java:1719)", > > "\tat > org.apache.flink.kinesis.shaded.com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.invoke(AWSSecurityTokenServiceClient.java:1686)", > > "\tat > org.apache.flink.kinesis.shaded.com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.invoke(AWSSecurityTokenServiceClient.java:1675)", > > "\tat > org.apache.flink.kinesis.shaded.com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.executeAssumeRole(AWSSecurityTokenServiceClient.java:589)", > > "\tat > org.apache.flink.kinesis.shaded.com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.assumeRole(AWSSecurityTokenServiceClient.java:561)", > > "\tat > org.apache.flink.kinesis.shaded.com.amazonaws.auth.STSAssumeRoleSessionCredentialsProvider.newSession(STSAssumeRoleSessionCredentialsProvider.java:321)", > > "\tat > org.apache.flink.kinesis.shaded.com.amazonaws.auth.STSAssumeRoleSessionCredentialsProvider.access$000(STSAssumeRoleSessionCredentialsProvider.java:37)", > > "\tat > org.apache.flink.kinesis.shaded.com.amazonaws.auth.STSAssumeRoleSessionCredentialsProvider$1.call(STSAssumeRoleSessionCredentialsProvider.java:76)", > > "\tat > org.apache.flink.kinesis.shaded.com.amazonaws.auth.STSAssumeRoleSessionCredentialsProvider$1.call(STSAssumeRoleSessionCredentialsProvider.java:73)", > > "\tat > org.apache.flink.kinesis.shaded.com.amazonaws.auth.RefreshableTask.refreshValue(RefreshableTask.java:257)", > > "\tat > org.apache.flink.kinesis.shaded.com.amazonaws.auth.RefreshableTask.blockingRefresh(RefreshableTask.java:213)", > > "\tat > org.apache.flink.kinesis.shaded.com.amazonaws.auth.RefreshableTask.getValue(RefreshableTask.java:154)", > > "\tat > org.apache.flink.kinesis.shaded.com.amazonaws.auth.STSAssumeRoleSessionCredentialsProvider.getCredentials(STSAssumeRoleSessionCredentialsProvider.java:299)", > > "\tat > org.apache.flink.kinesis.shaded.com.amazonaws.auth.STSAssumeRoleSessionCredentialsProvider.getCredentials(STSAssumeRoleSessionCredentialsProvider.java:36)", > > "\tat > org.apache.flink.kinesis.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.getCredentialsFromContext(AmazonHttpClient.java:1251)", > > "\tat > org.apache.flink.kinesis.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.runBeforeRequestHandlers(AmazonHttpClient.java:827)", > > "\tat > org.apache.flink.kinesis.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:777)", > > "\tat > org.apache.flink.kinesis.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:764)", > > "\tat > org.apache.flink.kinesis.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:738)", > > "\tat > org.apache.flink.kinesis.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:698)", > > "\tat > org.apache.flink.kinesis.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:680)", > > "\tat > org.apache.flink.kinesis.shaded.com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:544)", > > "\tat > org.apache.flink.kinesis.shaded.com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:524)", > > "\tat > org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.AmazonKinesisClient.doInvoke(AmazonKinesisClient.java:2809)", > > "\tat > org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.AmazonKinesisClient.invoke(AmazonKinesisClient.java:2776)", > > "\tat > org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.AmazonKinesisClient.invoke(AmazonKinesisClient.java:2765)", > > "\tat > org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.AmazonKinesisClient.executeListShards(AmazonKinesisClient.java:1557)", > > "\tat > org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.AmazonKinesisClient.listShards(AmazonKinesisClient.java:1528)", > > "\tat > org.apache.flink.streaming.connectors.kinesis.proxy.KinesisProxy.listShards(KinesisProxy.java:442)", > > "\tat > org.apache.flink.streaming.connectors.kinesis.proxy.KinesisProxy.getShardsOfStream(KinesisProxy.java:392)", > > "\tat > org.apache.flink.streaming.connectors.kinesis.proxy.KinesisProxy.getShardList(KinesisProxy.java:282)", > > "\tat > org.apache.flink.streaming.connectors.kinesis.internals.KinesisDataFetcher.discoverNewShardsToSubscribe(KinesisDataFetcher.java:681)", > > "\tat > org.apache.flink.streaming.connectors.kinesis.FlinkKinesisConsumer.run(FlinkKinesisConsumer.java:287)", > > "\tat > org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:93)", > > "\tat > org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:57)", > > "\tat > org.apache.flink.streaming.runtime.tasks.SourceStreamTask.run(SourceStreamTask.java:97)", > > "\tat > org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:308)", > > "\tat > org.apache.flink.runtime.taskmanager.Task.run(Task.java:714)", > > "\tat java.lang.Thread.run(Thread.java:748)", > > * "Caused by: java.lang.ClassCastException: > com.ctc.wstx.stax.WstxInputFactory cannot be cast to > javax.xml.stream.XMLInputFactory",* > > "\tat > javax.xml.stream.XMLInputFactory.newInstance(XMLInputFactory.java:41)", > > "\tat > org.apache.flink.kinesis.shaded.com.amazonaws.util.XmlUtils.createXmlInputFactory(XmlUtils.java:63)", > > "\tat > org.apache.flink.kinesis.shaded.com.amazonaws.util.XmlUtils.access$000(XmlUtils.java:27)", > > "\tat > org.apache.flink.kinesis.shaded.com.amazonaws.util.XmlUtils$1.initialValue(XmlUtils.java:36)", > > "\tat > org.apache.flink.kinesis.shaded.com.amazonaws.util.XmlUtils$1.initialValue(XmlUtils.java:33)", > > "\tat java.lang.ThreadLocal.setInitialValue(ThreadLocal.java:180)", > > "\tat java.lang.ThreadLocal.get(ThreadLocal.java:170)", > > "\tat > org.apache.flink.kinesis.shaded.com.amazonaws.util.XmlUtils.getXmlInputFactory(XmlUtils.java:54)", > > "\tat > org.apache.flink.kinesis.shaded.com.amazonaws.http.StaxResponseHandler.handle(StaxResponseHandler.java:94)", > > "\tat > org.apache.flink.kinesis.shaded.com.amazonaws.http.StaxResponseHandler.handle(StaxResponseHandler.java:42)", > > "\tat > org.apache.flink.kinesis.shaded.com.amazonaws.http.response.AwsResponseHandlerAdapter.handle(AwsResponseHandlerAdapter.java:69)", > > "\tat > org.apache.flink.kinesis.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleResponse(AmazonHttpClient.java:1714)", > > "\t... 49 more" > > ] > > > -- Arvid Heise | Senior Java Developer <https://www.ververica.com/> Follow us @VervericaData -- Join Flink Forward <https://flink-forward.org/> - The Apache Flink Conference Stream Processing | Event Driven | Real Time -- Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany -- Ververica GmbH Registered at Amtsgericht Charlottenburg: HRB 158244 B Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji (Toni) Cheng
