[
https://issues.apache.org/jira/browse/FLINK-8720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16377835#comment-16377835
]
dejan miljkovic commented on FLINK-8720:
----------------------------------------
Thanks for the suggestions. Removing Hadoop dependencies from jar did solve the
problem!!!
The application is built on Flink 1.4.1. It looks that more work is needed for
class loading logic. I noticed that in application works in InteligJ with some
versions of jars but does not work when submitted to cluster.
I tried "parent-first" option but that required adding more dependencies in
pom.xml. Did not proceed with this because I would affect other application
that are deployed.
One more thanks a lot for the response and solution.
> Logging exception with S3 connector and BucketingSink
> -----------------------------------------------------
>
> Key: FLINK-8720
> URL: https://issues.apache.org/jira/browse/FLINK-8720
> Project: Flink
> Issue Type: Bug
> Components: Streaming Connectors
> Affects Versions: 1.4.1
> Reporter: dejan miljkovic
> Priority: Critical
>
> Trying to stream data to S3. Code works from InteliJ. When submitting code
> trough UI on my machine (single node cluster started by start-cluster.sh
> script) below stack trace is produced.
>
> Below is the link to the simple test app that is streaming data to S3.
> [https://github.com/dmiljkovic/test-flink-bucketingsink-s3]
> The behavior is bit different but same error is produced. Job works only
> once. If job is submitted second time below stack trace is produced. If I
> restart the cluster job works but only for the first time.
>
>
> org.apache.commons.logging.LogConfigurationException:
> java.lang.IllegalAccessError:
> org/apache/commons/logging/impl/LogFactoryImpl$3 (Caused by
> java.lang.IllegalAccessError:
> org/apache/commons/logging/impl/LogFactoryImpl$3)
> at
> org.apache.commons.logging.impl.LogFactoryImpl.newInstance(LogFactoryImpl.java:637)
> at
> org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImpl.java:336)
> at
> org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImpl.java:310)
> at org.apache.commons.logging.LogFactory.getLog(LogFactory.java:685)
> at
> org.apache.http.impl.conn.PoolingClientConnectionManager.<init>(PoolingClientConnectionManager.java:76)
> at
> org.apache.http.impl.conn.PoolingClientConnectionManager.<init>(PoolingClientConnectionManager.java:102)
> at
> org.apache.http.impl.conn.PoolingClientConnectionManager.<init>(PoolingClientConnectionManager.java:88)
> at
> org.apache.http.impl.conn.PoolingClientConnectionManager.<init>(PoolingClientConnectionManager.java:96)
> at
> com.amazonaws.http.ConnectionManagerFactory.createPoolingClientConnManager(ConnectionManagerFactory.java:26)
> at
> com.amazonaws.http.HttpClientFactory.createHttpClient(HttpClientFactory.java:96)
> at com.amazonaws.http.AmazonHttpClient.<init>(AmazonHttpClient.java:158)
> at
> com.amazonaws.AmazonWebServiceClient.<init>(AmazonWebServiceClient.java:119)
> at
> com.amazonaws.services.s3.AmazonS3Client.<init>(AmazonS3Client.java:389)
> at
> com.amazonaws.services.s3.AmazonS3Client.<init>(AmazonS3Client.java:371)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.initialize(S3AFileSystem.java:235)
> at
> org.apache.flink.streaming.connectors.fs.bucketing.BucketingSink.createHadoopFileSystem(BucketingSink.java:1206)
> at
> org.apache.flink.streaming.connectors.fs.bucketing.BucketingSink.initFileSystem(BucketingSink.java:411)
> at
> org.apache.flink.streaming.connectors.fs.bucketing.BucketingSink.initializeState(BucketingSink.java:355)
> at
> org.apache.flink.streaming.util.functions.StreamingFunctionUtils.tryRestoreFunction(StreamingFunctionUtils.java:178)
> at
> org.apache.flink.streaming.util.functions.StreamingFunctionUtils.restoreFunctionState(StreamingFunctionUtils.java:160)
> at
> org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.initializeState(AbstractUdfStreamOperator.java:96)
> at
> org.apache.flink.streaming.api.operators.AbstractStreamOperator.initializeState(AbstractStreamOperator.java:258)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.initializeOperators(StreamTask.java:694)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.initializeState(StreamTask.java:682)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:253)
> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:718)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.IllegalAccessError:
> org/apache/commons/logging/impl/LogFactoryImpl$3
> at
> org.apache.commons.logging.impl.LogFactoryImpl.getParentClassLoader(LogFactoryImpl.java:700)
> at
> org.apache.commons.logging.impl.LogFactoryImpl.createLogFromClass(LogFactoryImpl.java:1187)
> at
> org.apache.commons.logging.impl.LogFactoryImpl.discoverLogImplementation(LogFactoryImpl.java:914)
> at
> org.apache.commons.logging.impl.LogFactoryImpl.newInstance(LogFactoryImpl.java:604)
> ... 26 more
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
