L1nq0 opened a new issue, #9077: URL: https://github.com/apache/storm/issues/9077
The #9076 review raised a design question that deserves its own issue. Today DeserializingConnectionCallback.recv() decides to drop a message by matching the decode failure against a list of exception types, walked along the cause chain. The review sketched a cleaner shape: KryoTupleDeserializer would validate the structural fields it already reads (source task id, stream id) and throw a dedicated exception for values the topology cannot resolve, instead of relying on a NullPointerException or IllegalArgumentException coming from deep in the lookup; the exception types that only a corrupt byte stream produces (KryoException, IOException, BufferUnderflowException, NegativeArraySizeException) would be wrapped into that same dedicated exception at the source; and recv() would catch exactly one intentional type, with everything else propagating. The open problem is the wrap boundary. kryo.deserializeFrom runs the framework decode and the serializers registered through topology.kryo.register with nothing between them, so an IllegalArgumentException or ClassCastException thrown by a user serializer is indistinguishable from the same type thrown by the framework decode. Dropping both swallows real topology bugs. Unwrapping both is not possible either: with registration required, an unregistered class fails with a raw IllegalArgumentException from Kryo.getRegistration(Class), which is exactly the remote worker-kill vector #9074 fixed, so the generic types cannot simply be made fatal again. Directions worth considering: structural validation typed at the source as above, plus a strict mode that turns drops back into fatal errors for operators who prefer loud failure; or attaching enough context (task, stream, input position) to the wrapped failure to separate framework decode errors from serializer errors in the logs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
