WangYu created FLINK-24309:
------------------------------
Summary: NoClassDefFoundError occurs when using spring-data-redis
for event notification in a flink cluster environment
Key: FLINK-24309
URL: https://issues.apache.org/jira/browse/FLINK-24309
Project: Flink
Issue Type: Bug
Components: API / DataStream
Affects Versions: 1.13.0
Environment: Flink 1.13.0 zookeeper cluster
1 JobManager
3 TaskManagers
Every TaskManager has 8 slots
And parallelism of every subtask is 6
Reporter: WangYu
I used spring-data-redis in my Flink application to implement notifications for
subscribing to refresh events. The code is as follows:
@Bean
public RedisMessageListenerContainer redisContainer(RedisConnectionFactory
redisConnectionFactory, MessageListenerAdapter commonListenerAdapter) {
RedisMessageListenerContainer container = new
RedisMessageListenerContainer();
container.setConnectionFactory(redisConnectionFactory);
container.addMessageListener(commonListenerAdapter, new
ChannelTopic(ApplicationConstants.REDIS_TOPIC_NAME));
return container;
}
@Bean
MessageListenerAdapter commonListenerAdapter(RedisReceiver redisReceiver) {
MessageListenerAdapter messageListenerAdapter = new
MessageListenerAdapter(redisReceiver, "onMessage");
messageListenerAdapter.setSerializer(jacksonSerializer());
return messageListenerAdapter;
}
It works very well in the local environment, application can receive
notifications of refresh events through redis.
But when I deployed the application to the Flink cluster, I got an error
message:
2021-09-16 16:22:05,197 WARN io.netty.channel.AbstractChannelHandlerContext
[] - An exception 'java.lang.NoClassDefFoundError:
io/netty/util/internal/logging/InternalLogLevel' [enable DEBUG level for full
stacktrace] was thrown by a user handler's exceptionCaught() method while
handling the following exception:2021-09-16 16:22:05,197 WARN
io.netty.channel.AbstractChannelHandlerContext [] - An exception
'java.lang.NoClassDefFoundError:
io/netty/util/internal/logging/InternalLogLevel' [enable DEBUG level for full
stacktrace] was thrown by a user handler's exceptionCaught() method while
handling the following exception:java.lang.NoClassDefFoundError:
org/springframework/data/redis/connection/DefaultMessage at
org.springframework.data.redis.connection.lettuce.LettuceMessageListener.message(LettuceMessageListener.java:43)
~[blob_p-e4b9f89a7dac271b36297079ae0306cc0273f659-a280a62d8bc228a1b65e682dcf2f8472:?]
at
org.springframework.data.redis.connection.lettuce.LettuceMessageListener.message(LettuceMessageListener.java:29)
~[blob_p-e4b9f89a7dac271b36297079ae0306cc0273f659-a280a62d8bc228a1b65e682dcf2f8472:?]
at
io.lettuce.core.pubsub.PubSubEndpoint.notifyListeners(PubSubEndpoint.java:191)
~[blob_p-e4b9f89a7dac271b36297079ae0306cc0273f659-a280a62d8bc228a1b65e682dcf2f8472:?]
at
io.lettuce.core.pubsub.PubSubEndpoint.notifyMessage(PubSubEndpoint.java:180)
~[blob_p-e4b9f89a7dac271b36297079ae0306cc0273f659-a280a62d8bc228a1b65e682dcf2f8472:?]
at
io.lettuce.core.pubsub.PubSubCommandHandler.doNotifyMessage(PubSubCommandHandler.java:217)
~[blob_p-e4b9f89a7dac271b36297079ae0306cc0273f659-a280a62d8bc228a1b65e682dcf2f8472:?]
at
io.lettuce.core.pubsub.PubSubCommandHandler.decode(PubSubCommandHandler.java:123)
~[blob_p-e4b9f89a7dac271b36297079ae0306cc0273f659-a280a62d8bc228a1b65e682dcf2f8472:?]
at
io.lettuce.core.protocol.CommandHandler.channelRead(CommandHandler.java:591)
~[blob_p-e4b9f89a7dac271b36297079ae0306cc0273f659-a280a62d8bc228a1b65e682dcf2f8472:?]
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
[blob_p-e4b9f89a7dac271b36297079ae0306cc0273f659-a280a62d8bc228a1b65e682dcf2f8472:?]
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
[blob_p-e4b9f89a7dac271b36297079ae0306cc0273f659-a280a62d8bc228a1b65e682dcf2f8472:?]
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
[blob_p-e4b9f89a7dac271b36297079ae0306cc0273f659-a280a62d8bc228a1b65e682dcf2f8472:?]
at
io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
[blob_p-e4b9f89a7dac271b36297079ae0306cc0273f659-a280a62d8bc228a1b65e682dcf2f8472:?]
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
[blob_p-e4b9f89a7dac271b36297079ae0306cc0273f659-a280a62d8bc228a1b65e682dcf2f8472:?]
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
[blob_p-e4b9f89a7dac271b36297079ae0306cc0273f659-a280a62d8bc228a1b65e682dcf2f8472:?]
at
io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
[blob_p-e4b9f89a7dac271b36297079ae0306cc0273f659-a280a62d8bc228a1b65e682dcf2f8472:?]
at
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
[blob_p-e4b9f89a7dac271b36297079ae0306cc0273f659-a280a62d8bc228a1b65e682dcf2f8472:?]
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
[blob_p-e4b9f89a7dac271b36297079ae0306cc0273f659-a280a62d8bc228a1b65e682dcf2f8472:?]
at
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
[blob_p-e4b9f89a7dac271b36297079ae0306cc0273f659-a280a62d8bc228a1b65e682dcf2f8472:?]
at
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
[blob_p-e4b9f89a7dac271b36297079ae0306cc0273f659-a280a62d8bc228a1b65e682dcf2f8472:?]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
[blob_p-e4b9f89a7dac271b36297079ae0306cc0273f659-a280a62d8bc228a1b65e682dcf2f8472:?]
at
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
[blob_p-e4b9f89a7dac271b36297079ae0306cc0273f659-a280a62d8bc228a1b65e682dcf2f8472:?]
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
[blob_p-e4b9f89a7dac271b36297079ae0306cc0273f659-a280a62d8bc228a1b65e682dcf2f8472:?]
at
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
[blob_p-e4b9f89a7dac271b36297079ae0306cc0273f659-a280a62d8bc228a1b65e682dcf2f8472:?]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_271]Caused by:
java.lang.ClassNotFoundException:
org.springframework.data.redis.connection.DefaultMessage at
java.net.URLClassLoader.findClass(URLClassLoader.java:382) ~[?:1.8.0_271] at
java.lang.ClassLoader.loadClass(ClassLoader.java:418) ~[?:1.8.0_271] at
org.apache.flink.util.FlinkUserCodeClassLoader.loadClassWithoutExceptionHandling(FlinkUserCodeClassLoader.java:64)
~[flink-dist_2.11-1.13.0.jar:1.13.0] at
org.apache.flink.util.ChildFirstClassLoader.loadClassWithoutExceptionHandling(ChildFirstClassLoader.java:74)
~[flink-dist_2.11-1.13.0.jar:1.13.0] at
org.apache.flink.util.FlinkUserCodeClassLoader.loadClass(FlinkUserCodeClassLoader.java:48)
~[flink-dist_2.11-1.13.0.jar:1.13.0] at
java.lang.ClassLoader.loadClass(ClassLoader.java:351) ~[?:1.8.0_271] ... 23
more2021-09-16 16:22:05,215 INFO
com.cccar.vdm.mpp.client.listener.ModelInstanceRefreshListener [] -
============= refresh model instance =============
I checked the jar package of the application and confirmed that the
rg/springframework/data/redis/connection/DefaultMessage class file is in the
classpath.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
