clintropolis opened a new issue #7252: Kafka Tasks unable restore persisted segments properly URL: https://github.com/apache/incubator-druid/issues/7252 ### Affected Version All versions after #6431 ### Description Kakfa indexing tasks (likely kinesis as well) are unable to restore properly if segments have been incrementally persisted because they cannot deserialize `SequenceMetadata`, resulting in an error in the form of: ``` 2019-03-12T02:18:50,238 ERROR [task-runner-0-priority-0] org.apache.druid.indexing.seekablestream.SeekableStreamIndexTaskRunner - Encountered exception while running task. com.fasterxml.jackson.databind.JsonMappingException: Unrecognized Type: [null] at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCache2(DeserializerCache.java:269) ~[jackson-databind-2.6.7.jar:2.6.7] at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCacheValueDeserializer(DeserializerCache.java:244) ~[jackson-databind-2.6.7.jar:2.6.7] at com.fasterxml.jackson.databind.deser.DeserializerCache.findValueDeserializer(DeserializerCache.java:142) ~[jackson-databind-2.6.7.jar:2.6.7] at com.fasterxml.jackson.databind.DeserializationContext.findContextualValueDeserializer(DeserializationContext.java:428) ~[jackson-databind-2.6.7.jar:2.6.7] at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.createContextual(CollectionDeserializer.java:164) ~[jackson-databind-2.6.7.jar:2.6.7] at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.createContextual(CollectionDeserializer.java:25) ~[jackson-databind-2.6.7.jar:2.6.7] at com.fasterxml.jackson.databind.DeserializationContext.handleSecondaryContextualization(DeserializationContext.java:669) ~[jackson-databind-2.6.7.jar:2.6.7] at com.fasterxml.jackson.databind.DeserializationContext.findRootValueDeserializer(DeserializationContext.java:466) ~[jackson-databind-2.6.7.jar:2.6.7] at com.fasterxml.jackson.databind.ObjectMapper._findRootDeserializer(ObjectMapper.java:3838) ~[jackson-databind-2.6.7.jar:2.6.7] at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3732) ~[jackson-databind-2.6.7.jar:2.6.7] at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2639) ~[jackson-databind-2.6.7.jar:2.6.7] at org.apache.druid.indexing.seekablestream.SeekableStreamIndexTaskRunner.restoreSequences(SeekableStreamIndexTaskRunner.java:942) ~[druid-indexing-service-0.14.0-iap-pre3.jar:0.14.0-iap-pre3] ... ``` The issue is that `SequenceMetadata` is a private class using generic types defined in the outer class, `SeekableStreamIndexTaskRunner<PartitionIdType, SequenceOffsetType>`, which determine the map types which store offsets in the persisted sequence metadata json file. The file is written correctly, but the `TypeReference` given to the object mapper is not able to properly read it during deserialization, resulting in the above error. Fix underway, but untangling this is sort of messy unfortunately.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
