Hi, I could not find any open Jira for the problem you describe. Could you please open one?
Best, Stefan > Am 19.09.2018 um 09:54 schrieb Paul Lam <paullin3...@gmail.com>: > > Hi, > > I’m using StreamingFileSink of 1.6.0 to write logs to S3 and encounter a > classloader problem. It seems that there are conflicts in > flink-shaded-hadoop2-uber-1.6.0.jar and flink-s3-fs-hadoop-1.6.0.jar, and > maybe related to class loading orders. > > Did anyone meet this problem? Thanks a lot! > > The stack traces are as below: > > java.io.IOException: java.lang.RuntimeException: class > org.apache.hadoop.security.LdapGroupsMapping not > org.apache.flink.fs.s3hadoop.shaded.org.apache.hadoop.security.GroupMappingServiceProvider > at > org.apache.flink.fs.s3hadoop.shaded.org.apache.flink.runtime.fs.hdfs.AbstractFileSystemFactory.create(AbstractFileSystemFactory.java:62) > at > org.apache.flink.core.fs.FileSystem.getUnguardedFileSystem(FileSystem.java:395) > at org.apache.flink.core.fs.FileSystem.get(FileSystem.java:318) > at > org.apache.flink.streaming.api.functions.sink.filesystem.Buckets.<init>(Buckets.java:111) > at > org.apache.flink.streaming.api.functions.sink.filesystem.StreamingFileSink$RowFormatBuilder.createBuckets(StreamingFileSink.java:242) > at > org.apache.flink.streaming.api.functions.sink.filesystem.StreamingFileSink.initializeState(StreamingFileSink.java:327) > at > org.apache.flink.streaming.util.functions.StreamingFunctionUtils.tryRestoreFunction(StreamingFunctionUtils.java:178) > at > org.apache.flink.streaming.util.functions.StreamingFunctionUtils.restoreFunctionState(StreamingFunctionUtils.java:160) > at > org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.initializeState(AbstractUdfStreamOperator.java:96) > at > org.apache.flink.streaming.api.operators.AbstractStreamOperator.initializeState(AbstractStreamOperator.java:254) > at > org.apache.flink.streaming.runtime.tasks.StreamTask.initializeState(StreamTask.java:738) > at > org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:289) > at org.apache.flink.runtime.taskmanager.Task.run(Task.java:711) > at java.lang.Thread.run(Thread.java:748) > Caused by: java.lang.RuntimeException: java.lang.RuntimeException: class > org.apache.hadoop.security.LdapGroupsMapping not > org.apache.flink.fs.s3hadoop.shaded.org.apache.hadoop.security.GroupMappingServiceProvider > at > org.apache.flink.fs.s3hadoop.shaded.org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2246) > at > org.apache.flink.fs.s3hadoop.shaded.org.apache.hadoop.security.Groups.<init>(Groups.java:108) > at > org.apache.flink.fs.s3hadoop.shaded.org.apache.hadoop.security.Groups.<init>(Groups.java:102) > at > org.apache.flink.fs.s3hadoop.shaded.org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:450) > at > org.apache.flink.fs.s3hadoop.shaded.org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:309) > at > org.apache.flink.fs.s3hadoop.shaded.org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:276) > at > org.apache.flink.fs.s3hadoop.shaded.org.apache.hadoop.security.UserGroupInformation.loginUserFromSubject(UserGroupInformation.java:832) > at > org.apache.flink.fs.s3hadoop.shaded.org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:802) > at > org.apache.flink.fs.s3hadoop.shaded.org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:675) > at > org.apache.flink.fs.s3hadoop.shaded.org.apache.hadoop.fs.s3a.S3AFileSystem.initialize(S3AFileSystem.java:177) > at > org.apache.flink.fs.s3hadoop.shaded.org.apache.flink.runtime.fs.hdfs.AbstractFileSystemFactory.create(AbstractFileSystemFactory.java:57) > ... 13 more > Caused by: java.lang.RuntimeException: class > org.apache.hadoop.security.LdapGroupsMapping not > org.apache.flink.fs.s3hadoop.shaded.org.apache.hadoop.security.GroupMappingServiceProvider > at > org.apache.flink.fs.s3hadoop.shaded.org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2240) > ... 23 more > > > Best, > Paul Lam > > >
