[
https://issues.apache.org/jira/browse/STORM-3751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arwin S Tio updated STORM-3751:
-------------------------------
Description:
Hello,
I've recently upgraded to Storm 2.2.0 and have been getting this error:
{code:java}
2021-03-07 04:36:51.061 o.a.s.m.n.StormServerHandler
Netty-server-localhost-6700-worker-1 [ERROR] server errors in handling the
request
java.lang.NullPointerException: null
at
org.apache.storm.daemon.worker.WorkerState.transferLocalBatch(WorkerState.java:543)
~[storm-client-2.2.0.jar:2.2.0]
at
org.apache.storm.messaging.DeserializingConnectionCallback.recv(DeserializingConnectionCallback.java:71)
~[storm-client-2.2.0.jar:2.2.0]
at org.apache.storm.messaging.netty.Server.enqueue(Server.java:146)
~[storm-client-2.2.0.jar:2.2.0]
at org.apache.storm.messaging.netty.Server.received(Server.java:264)
~[storm-client-2.2.0.jar:2.2.0]
at
org.apache.storm.messaging.netty.StormServerHandler.channelRead(StormServerHandler.java:51)
~[storm-client-2.2.0.jar:2.2.0]
at
org.apache.storm.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
[storm-shaded-deps-2.2.0.jar:2.2.0]
at
org.apache.storm.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
[storm-shaded-deps-2.2.0.jar:2.2.0]
at
org.apache.storm.shade.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
[storm-shaded-deps-2.2.0.jar:2.2.0]
at
org.apache.storm.shade.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:323)
[storm-shaded-deps-2.2.0.jar:2.2.0]
at
org.apache.storm.shade.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:297)
[storm-shaded-deps-2.2.0.jar:2.2.0]
at
org.apache.storm.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
[storm-shaded-deps-2.2.0.jar:2.2.0]
at
org.apache.storm.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
[storm-shaded-deps-2.2.0.jar:2.2.0]
at
org.apache.storm.shade.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
[storm-shaded-deps-2.2.0.jar:2.2.0]
at
org.apache.storm.shade.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1434)
[storm-shaded-deps-2.2.0.jar:2.2.0]
at
org.apache.storm.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
[storm-shaded-deps-2.2.0.jar:2.2.0]
at
org.apache.storm.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
[storm-shaded-deps-2.2.0.jar:2.2.0]
at
org.apache.storm.shade.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:965)
[storm-shaded-deps-2.2.0.jar:2.2.0]
at
org.apache.storm.shade.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
[storm-shaded-deps-2.2.0.jar:2.2.0]
at
org.apache.storm.shade.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:644)
[storm-shaded-deps-2.2.0.jar:2.2.0]
at
org.apache.storm.shade.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:579)
[storm-shaded-deps-2.2.0.jar:2.2.0]
at
org.apache.storm.shade.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:496)
[storm-shaded-deps-2.2.0.jar:2.2.0]
at
org.apache.storm.shade.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:458)
[storm-shaded-deps-2.2.0.jar:2.2.0]
at
org.apache.storm.shade.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:897)
[storm-shaded-deps-2.2.0.jar:2.2.0]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_272]
2021-03-07 04:36:51.061 o.a.s.m.n.StormServerHandler
Netty-server-localhost-6700-worker-1 [INFO] Received error in netty thread..
terminating server... {code}
This issue happens every 20-30 minutes and causes the workers to die/restart.
It seems related to https://issues.apache.org/jira/browse/STORM-3141 but seems
to have been fixed in 2.0.
I am happy to provide more information but at the moment am unsure of what is
relevant.
I have a suspicion that this is related to load-aware localOrShuffleGrouping
("LoadAwareShuffleGrouping") because this issue seems to have started when I
switched the Grouping, but again, not sure if it's actually related.
was:
Hello,
I've recently upgraded to Storm 2.2.0 and have been getting this error:
{code:java}
2021-03-07 04:36:51.061 o.a.s.m.n.StormServerHandler
Netty-server-localhost-6700-worker-1 [ERROR] server errors in handling the
request
java.lang.NullPointerException: null
at
org.apache.storm.daemon.worker.WorkerState.transferLocalBatch(WorkerState.java:543)
~[storm-client-2.2.0.jar:2.2.0]
at
org.apache.storm.messaging.DeserializingConnectionCallback.recv(DeserializingConnectionCallback.java:71)
~[storm-client-2.2.0.jar:2.2.0]
at org.apache.storm.messaging.netty.Server.enqueue(Server.java:146)
~[storm-client-2.2.0.jar:2.2.0]
at org.apache.storm.messaging.netty.Server.received(Server.java:264)
~[storm-client-2.2.0.jar:2.2.0]
at
org.apache.storm.messaging.netty.StormServerHandler.channelRead(StormServerHandler.java:51)
~[storm-client-2.2.0.jar:2.2.0]
at
org.apache.storm.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
[storm-shaded-deps-2.2.0.jar:2.2.0]
at
org.apache.storm.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
[storm-shaded-deps-2.2.0.jar:2.2.0]
at
org.apache.storm.shade.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
[storm-shaded-deps-2.2.0.jar:2.2.0]
at
org.apache.storm.shade.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:323)
[storm-shaded-deps-2.2.0.jar:2.2.0]
at
org.apache.storm.shade.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:297)
[storm-shaded-deps-2.2.0.jar:2.2.0]
at
org.apache.storm.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
[storm-shaded-deps-2.2.0.jar:2.2.0]
at
org.apache.storm.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
[storm-shaded-deps-2.2.0.jar:2.2.0]
at
org.apache.storm.shade.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
[storm-shaded-deps-2.2.0.jar:2.2.0]
at
org.apache.storm.shade.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1434)
[storm-shaded-deps-2.2.0.jar:2.2.0]
at
org.apache.storm.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
[storm-shaded-deps-2.2.0.jar:2.2.0]
at
org.apache.storm.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
[storm-shaded-deps-2.2.0.jar:2.2.0]
at
org.apache.storm.shade.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:965)
[storm-shaded-deps-2.2.0.jar:2.2.0]
at
org.apache.storm.shade.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
[storm-shaded-deps-2.2.0.jar:2.2.0]
at
org.apache.storm.shade.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:644)
[storm-shaded-deps-2.2.0.jar:2.2.0]
at
org.apache.storm.shade.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:579)
[storm-shaded-deps-2.2.0.jar:2.2.0]
at
org.apache.storm.shade.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:496)
[storm-shaded-deps-2.2.0.jar:2.2.0]
at
org.apache.storm.shade.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:458)
[storm-shaded-deps-2.2.0.jar:2.2.0]
at
org.apache.storm.shade.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:897)
[storm-shaded-deps-2.2.0.jar:2.2.0]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_272]
2021-03-07 04:36:51.061 o.a.s.m.n.StormServerHandler
Netty-server-localhost-6700-worker-1 [INFO] Received error in netty thread..
terminating server... {code}
It seems related to https://issues.apache.org/jira/browse/STORM-3141 but seems
to have been fixed in 2.0.
I am happy to provide more information but at the moment am unsure of what is
relevant.
I have a suspicion that this is related to load-aware localOrShuffleGrouping
("LoadAwareShuffleGrouping") because this issue seems to have started when I
switched the Grouping, but again, not sure if it's actually related.
> NPE in WorkerState.transferLocalBatch
> -------------------------------------
>
> Key: STORM-3751
> URL: https://issues.apache.org/jira/browse/STORM-3751
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-client
> Affects Versions: 2.2.0
> Reporter: Arwin S Tio
> Priority: Minor
>
> Hello,
>
> I've recently upgraded to Storm 2.2.0 and have been getting this error:
>
> {code:java}
> 2021-03-07 04:36:51.061 o.a.s.m.n.StormServerHandler
> Netty-server-localhost-6700-worker-1 [ERROR] server errors in handling the
> request
> java.lang.NullPointerException: null
> at
> org.apache.storm.daemon.worker.WorkerState.transferLocalBatch(WorkerState.java:543)
> ~[storm-client-2.2.0.jar:2.2.0]
> at
> org.apache.storm.messaging.DeserializingConnectionCallback.recv(DeserializingConnectionCallback.java:71)
> ~[storm-client-2.2.0.jar:2.2.0]
> at org.apache.storm.messaging.netty.Server.enqueue(Server.java:146)
> ~[storm-client-2.2.0.jar:2.2.0]
> at org.apache.storm.messaging.netty.Server.received(Server.java:264)
> ~[storm-client-2.2.0.jar:2.2.0]
> at
> org.apache.storm.messaging.netty.StormServerHandler.channelRead(StormServerHandler.java:51)
> ~[storm-client-2.2.0.jar:2.2.0]
> at
> org.apache.storm.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
> [storm-shaded-deps-2.2.0.jar:2.2.0]
> at
> org.apache.storm.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
> [storm-shaded-deps-2.2.0.jar:2.2.0]
> at
> org.apache.storm.shade.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
> [storm-shaded-deps-2.2.0.jar:2.2.0]
> at
> org.apache.storm.shade.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:323)
> [storm-shaded-deps-2.2.0.jar:2.2.0]
> at
> org.apache.storm.shade.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:297)
> [storm-shaded-deps-2.2.0.jar:2.2.0]
> at
> org.apache.storm.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
> [storm-shaded-deps-2.2.0.jar:2.2.0]
> at
> org.apache.storm.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
> [storm-shaded-deps-2.2.0.jar:2.2.0]
> at
> org.apache.storm.shade.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
> [storm-shaded-deps-2.2.0.jar:2.2.0]
> at
> org.apache.storm.shade.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1434)
> [storm-shaded-deps-2.2.0.jar:2.2.0]
> at
> org.apache.storm.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
> [storm-shaded-deps-2.2.0.jar:2.2.0]
> at
> org.apache.storm.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
> [storm-shaded-deps-2.2.0.jar:2.2.0]
> at
> org.apache.storm.shade.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:965)
> [storm-shaded-deps-2.2.0.jar:2.2.0]
> at
> org.apache.storm.shade.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
> [storm-shaded-deps-2.2.0.jar:2.2.0]
> at
> org.apache.storm.shade.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:644)
> [storm-shaded-deps-2.2.0.jar:2.2.0]
> at
> org.apache.storm.shade.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:579)
> [storm-shaded-deps-2.2.0.jar:2.2.0]
> at
> org.apache.storm.shade.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:496)
> [storm-shaded-deps-2.2.0.jar:2.2.0]
> at
> org.apache.storm.shade.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:458)
> [storm-shaded-deps-2.2.0.jar:2.2.0]
> at
> org.apache.storm.shade.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:897)
> [storm-shaded-deps-2.2.0.jar:2.2.0]
> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_272]
> 2021-03-07 04:36:51.061 o.a.s.m.n.StormServerHandler
> Netty-server-localhost-6700-worker-1 [INFO] Received error in netty thread..
> terminating server... {code}
>
> This issue happens every 20-30 minutes and causes the workers to die/restart.
> It seems related to https://issues.apache.org/jira/browse/STORM-3141 but
> seems to have been fixed in 2.0.
> I am happy to provide more information but at the moment am unsure of what is
> relevant.
> I have a suspicion that this is related to load-aware localOrShuffleGrouping
> ("LoadAwareShuffleGrouping") because this issue seems to have started when I
> switched the Grouping, but again, not sure if it's actually related.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
