Hi, thanks for the reply. It was indeed the class loading issue and it’s introduced by latest version of package “aws-kinesisanalytics-runtime”. I resolved the issue by removing the package and customized the runtime myself.
-- Best wishes Fu Kai From: Arvid Heise <ar...@ververica.com> Date: Thursday, April 23, 2020 at 9:03 PM To: "Fu, Kai" <k...@amazon.com> Cc: Chesnay Schepler <ches...@apache.org>, "user@flink.apache.org" <user@flink.apache.org> Subject: RE: [EXTERNAL] Unable to unmarshall response (com.ctc.wstx.stax.WstxInputFactory cannot be cast to javax.xml.stream.XMLInputFactory) CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe. This looks like a typical issue with classloading. kinesis is probably residing in flink-dist/lib while woodstock is added in your job.jar (or vice versa). Could you try to use both jars in the same way? Alternatively, could you provide more information regarding your dependencies? On Tue, Apr 21, 2020 at 11:21 AM Fu, Kai <k...@amazon.com<mailto:k...@amazon.com>> wrote: Hi, I’m using Flink 1.8 with JDK 8. -- Best wishes Fu Kai From: Chesnay Schepler <ches...@apache.org<mailto:ches...@apache.org>> Date: Tuesday, April 21, 2020 at 5:15 PM To: "Fu, Kai" <k...@amazon.com<mailto:k...@amazon.com>>, "user@flink.apache.org<mailto:user@flink.apache.org>" <user@flink.apache.org<mailto:user@flink.apache.org>> Subject: RE: [EXTERNAL] Unable to unmarshall response (com.ctc.wstx.stax.WstxInputFactory cannot be cast to javax.xml.stream.XMLInputFactory) Which Flink version are you using? On 21/04/2020 11:11, Fu, Kai wrote: Hi, I’m running Flink application on AWS Kinesis Flink platform to read a kinesis stream from another account with assumed role, while I’m getting exception like below. But it works when I’m running the application locally, I’ve given all the related roles admin permission. Could anyone help what’s the potential problem? [ "org.apache.flink.kinesis.shaded.com.amazonaws.SdkClientException: Unable to unmarshall response (com.ctc.wstx.stax.WstxInputFactory cannot be cast to javax.xml.stream.XMLInputFactory). Response Code: 200, Response Text: OK", "\tat org.apache.flink.kinesis.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleResponse(AmazonHttpClient.java:1738)", "\tat org.apache.flink.kinesis.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleSuccessResponse(AmazonHttpClient.java:1434)", "\tat org.apache.flink.kinesis.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1356)", "\tat org.apache.flink.kinesis.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1139)", "\tat org.apache.flink.kinesis.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:796)", "\tat org.apache.flink.kinesis.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:764)", "\tat org.apache.flink.kinesis.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:738)", "\tat org.apache.flink.kinesis.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:698)", "\tat org.apache.flink.kinesis.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:680)", "\tat org.apache.flink.kinesis.shaded.com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:544)", "\tat org.apache.flink.kinesis.shaded.com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:524)", "\tat org.apache.flink.kinesis.shaded.com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.doInvoke(AWSSecurityTokenServiceClient.java:1719)", "\tat org.apache.flink.kinesis.shaded.com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.invoke(AWSSecurityTokenServiceClient.java:1686)", "\tat org.apache.flink.kinesis.shaded.com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.invoke(AWSSecurityTokenServiceClient.java:1675)", "\tat org.apache.flink.kinesis.shaded.com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.executeAssumeRole(AWSSecurityTokenServiceClient.java:589)", "\tat org.apache.flink.kinesis.shaded.com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.assumeRole(AWSSecurityTokenServiceClient.java:561)", "\tat org.apache.flink.kinesis.shaded.com.amazonaws.auth.STSAssumeRoleSessionCredentialsProvider.newSession(STSAssumeRoleSessionCredentialsProvider.java:321)", "\tat org.apache.flink.kinesis.shaded.com.amazonaws.auth.STSAssumeRoleSessionCredentialsProvider.access$000(STSAssumeRoleSessionCredentialsProvider.java:37)", "\tat org.apache.flink.kinesis.shaded.com.amazonaws.auth.STSAssumeRoleSessionCredentialsProvider$1.call(STSAssumeRoleSessionCredentialsProvider.java:76)", "\tat org.apache.flink.kinesis.shaded.com.amazonaws.auth.STSAssumeRoleSessionCredentialsProvider$1.call(STSAssumeRoleSessionCredentialsProvider.java:73)", "\tat org.apache.flink.kinesis.shaded.com.amazonaws.auth.RefreshableTask.refreshValue(RefreshableTask.java:257)", "\tat org.apache.flink.kinesis.shaded.com.amazonaws.auth.RefreshableTask.blockingRefresh(RefreshableTask.java:213)", "\tat org.apache.flink.kinesis.shaded.com.amazonaws.auth.RefreshableTask.getValue(RefreshableTask.java:154)", "\tat org.apache.flink.kinesis.shaded.com.amazonaws.auth.STSAssumeRoleSessionCredentialsProvider.getCredentials(STSAssumeRoleSessionCredentialsProvider.java:299)", "\tat org.apache.flink.kinesis.shaded.com.amazonaws.auth.STSAssumeRoleSessionCredentialsProvider.getCredentials(STSAssumeRoleSessionCredentialsProvider.java:36)", "\tat org.apache.flink.kinesis.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.getCredentialsFromContext(AmazonHttpClient.java:1251)", "\tat org.apache.flink.kinesis.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.runBeforeRequestHandlers(AmazonHttpClient.java:827)", "\tat org.apache.flink.kinesis.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:777)", "\tat org.apache.flink.kinesis.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:764)", "\tat org.apache.flink.kinesis.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:738)", "\tat org.apache.flink.kinesis.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:698)", "\tat org.apache.flink.kinesis.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:680)", "\tat org.apache.flink.kinesis.shaded.com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:544)", "\tat org.apache.flink.kinesis.shaded.com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:524)", "\tat org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.AmazonKinesisClient.doInvoke(AmazonKinesisClient.java:2809)", "\tat org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.AmazonKinesisClient.invoke(AmazonKinesisClient.java:2776)", "\tat org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.AmazonKinesisClient.invoke(AmazonKinesisClient.java:2765)", "\tat org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.AmazonKinesisClient.executeListShards(AmazonKinesisClient.java:1557)", "\tat org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.AmazonKinesisClient.listShards(AmazonKinesisClient.java:1528)", "\tat org.apache.flink.streaming.connectors.kinesis.proxy.KinesisProxy.listShards(KinesisProxy.java:442)", "\tat org.apache.flink.streaming.connectors.kinesis.proxy.KinesisProxy.getShardsOfStream(KinesisProxy.java:392)", "\tat org.apache.flink.streaming.connectors.kinesis.proxy.KinesisProxy.getShardList(KinesisProxy.java:282)", "\tat org.apache.flink.streaming.connectors.kinesis.internals.KinesisDataFetcher.discoverNewShardsToSubscribe(KinesisDataFetcher.java:681)", "\tat org.apache.flink.streaming.connectors.kinesis.FlinkKinesisConsumer.run(FlinkKinesisConsumer.java:287)", "\tat org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:93)", "\tat org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:57)", "\tat org.apache.flink.streaming.runtime.tasks.SourceStreamTask.run(SourceStreamTask.java:97)", "\tat org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:308)", "\tat org.apache.flink.runtime.taskmanager.Task.run(Task.java:714)", "\tat java.lang.Thread.run(Thread.java:748)", "Caused by: java.lang.ClassCastException: com.ctc.wstx.stax.WstxInputFactory cannot be cast to javax.xml.stream.XMLInputFactory", "\tat javax.xml.stream.XMLInputFactory.newInstance(XMLInputFactory.java:41)", "\tat org.apache.flink.kinesis.shaded.com.amazonaws.util.XmlUtils.createXmlInputFactory(XmlUtils.java:63)", "\tat org.apache.flink.kinesis.shaded.com.amazonaws.util.XmlUtils.access$000(XmlUtils.java:27)", "\tat org.apache.flink.kinesis.shaded.com.amazonaws.util.XmlUtils$1.initialValue(XmlUtils.java:36)", "\tat org.apache.flink.kinesis.shaded.com.amazonaws.util.XmlUtils$1.initialValue(XmlUtils.java:33)", "\tat java.lang.ThreadLocal.setInitialValue(ThreadLocal.java:180)", "\tat java.lang.ThreadLocal.get(ThreadLocal.java:170)", "\tat org.apache.flink.kinesis.shaded.com.amazonaws.util.XmlUtils.getXmlInputFactory(XmlUtils.java:54)", "\tat org.apache.flink.kinesis.shaded.com.amazonaws.http.StaxResponseHandler.handle(StaxResponseHandler.java:94)", "\tat org.apache.flink.kinesis.shaded.com.amazonaws.http.StaxResponseHandler.handle(StaxResponseHandler.java:42)", "\tat org.apache.flink.kinesis.shaded.com.amazonaws.http.response.AwsResponseHandlerAdapter.handle(AwsResponseHandlerAdapter.java:69)", "\tat org.apache.flink.kinesis.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleResponse(AmazonHttpClient.java:1714)", "\t... 49 more" ] -- Arvid Heise | Senior Java Developer [Image removed by sender.]<https://www.ververica.com/> Follow us @VervericaData -- Join Flink Forward<https://flink-forward.org/> - The Apache Flink Conference Stream Processing | Event Driven | Real Time -- Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany -- Ververica GmbH Registered at Amtsgericht Charlottenburg: HRB 158244 B Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji (Toni) Cheng
