[
https://issues.apache.org/jira/browse/STORM-3582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17074497#comment-17074497
]
Simon Cooper commented on STORM-3582:
-------------------------------------
Another exception which may indicate it's a problem in kryo (this is when
serializing an ack tuple):
{code}java.lang.ArrayIndexOutOfBoundsException: arraycopy: last destination
index 32 out of bounds for byte[31]
at java.lang.System.arraycopy(Native Method) ~[?:?]
at com.esotericsoftware.kryo.io.Output.toBytes(Output.java:130)
~[kryo-shaded-3.0.3.jar:?]
at
org.apache.storm.serialization.KryoTupleSerializer.serialize(KryoTupleSerializer.java:41)
~[storm-client-2.1.0.jar:2.1.0]
at
org.apache.storm.daemon.worker.WorkerState.checkSerialize(WorkerState.java:580)
~[storm-client-2.1.0.jar:2.1.0]
at
org.apache.storm.executor.ExecutorTransfer.tryTransferLocal(ExecutorTransfer.java:99)
~[storm-client-2.1.0.jar:2.1.0]
at
org.apache.storm.executor.ExecutorTransfer.tryTransfer(ExecutorTransfer.java:64)
~[storm-client-2.1.0.jar:2.1.0]
at
org.apache.storm.executor.LocalExecutor$1.tryTransfer(LocalExecutor.java:36)
~[storm-client-2.1.0.jar:2.1.0]
at org.apache.storm.daemon.Task.sendUnanchored(Task.java:211)
~[storm-client-2.1.0.jar:2.1.0]
at
org.apache.storm.executor.bolt.BoltOutputCollectorImpl.ack(BoltOutputCollectorImpl.java:128)
~[storm-client-2.1.0.jar:2.1.0]
at org.apache.storm.task.OutputCollector.ack(OutputCollector.java:181)
~[storm-client-2.1.0.jar:2.1.0]{code}
> Kryo errors when using custom serialization
> -------------------------------------------
>
> Key: STORM-3582
> URL: https://issues.apache.org/jira/browse/STORM-3582
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-core
> Affects Versions: 2.1.0
> Reporter: Julien Nioche
> Priority: Blocker
>
> (this has been reported on the user list in January and previously by a
> different user)
>
> ------------------------------------------------------------------------------------
>
> My code works fine with Storm 1.x but the workers crash constantly with Storm
> 2.x
>
> Some exceptions look like
>
> {{_com.esotericsoftware.kryo.KryoException: Buffer underflow._}}
> {{_at com.esotericsoftware.kryo.io.Input.require(Input.java:199)
> ~[kryo-3.0.3.jar:?]_}}
> {{_at com.esotericsoftware.kryo.io.Input.readUtf8_slow(Input.java:575)
> ~[kryo-3.0.3.jar:?]_}}
> {{_at com.esotericsoftware.kryo.io.Input.readUtf8(Input.java:553)
> ~[kryo-3.0.3.jar:?]_}}
> {{_at com.esotericsoftware.kryo.io.Input.readString(Input.java:483)
> ~[kryo-3.0.3.jar:?]_}}
> {{whereas others are}}
>
> {{_2020-01-21 11:13:39.368 o.a.s.m.n.StormServerHandler
> Netty-server-localhost-6701-worker-1 [ERROR] server errors in handling the
> request_}}
> {{_com.esotericsoftware.kryo.KryoException: Encountered unregistered class
> ID: *95*
> at
> com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:137)
> ~[kryo-3.0.3.jar:?]
> at com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:670) ~[kryo-3.0.3.jar:?]
> at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:781)
> ~[kryo-3.0.3.jar:?]
> at
> com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:153)
> ~[kryo-3.0.3.jar:?]
> at
> com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:39)
> ~[kryo-3.0.3.jar:?]
> at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:793)
> ~[kryo-3.0.3.jar:?]
> at
> com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:134)
> ~[kryo-3.0.3.jar:?]
> at
> com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:40)
> ~[kryo-3.0.3.jar:?]
> at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:689)
> ~[kryo-3.0.3.jar:?]
> at
> org.apache.storm.serialization.KryoValuesDeserializer.deserializeFrom(KryoValuesDeserializer.java:31)
> ~[storm-client-2.1.0.jar:2.1.0]_}}
> {{the class ID values are random integers. }}
> {{}}
> The tuples in my topologies contain mostly standard classes like String; the
> only exception is the following class
>
> [https://github.com/DigitalPebble/storm-crawler/blob/2.x/core/src/main/java/com/digitalpebble/stormcrawler/Metadata.java#L41]
>
> for which we specify a custom serialization for Kryo.
>
> My configurations contain
>
> | topology.kryo.register:|- com.digitalpebble.stormcrawler.Metadata|
> to register the custom class.
>
> The user who had reported the problem first also used custom serialization.
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
