druid-matt opened a new issue #10207:
URL: https://github.com/apache/druid/issues/10207
Reporting bug in the druid-time-min-max extenstion. It's returning
TimestampAggregatorFactory from getCombiningFactory() of both
TimestampMinAggregatorFactory and TimestampMaxAggregatorFactory. That's not
proper, because TimestampAggregatorFactory is not JSON-round-trippable, and in
general we expect the combining factories to be JSON-round-trippable.
User is having trouble with druid-time-min-max extension
He has metrics that looks like:
```
{
"type": "timeMin",
"name": "first_seen",
"fieldName": "timestamp"
},
{
"type": "timeMax",
"name": "last_seen",
"fieldName": "timestamp"
}
```
the extension seem to work generally but when he looks at the logs he
sometimes see this during ingestion - not every time but just some time:
```
20-07-22T18:30:12,176 WARN
[[index_kafka_REDACTED_22eff7a102317ab_jipjfaed]-appenderator-persist]
org.apache.druid.segment.IndexIO - Failed to load metadata for segment
[var/druid/task/index_kafka_REDACTED_22eff7a102317ab_jipjfaed/work/persist/REDACTED_2020-07-22T00:00:00.000Z_2020-07-23T00:00:00.000Z_2020-07-22T17:54:42.013Z/6]
com.fasterxml.jackson.databind.exc.InvalidTypeIdException: Could not resolve
type id 'TimestampAggregatorFactory' as a subtype of
`org.apache.druid.query.aggregation.AggregatorFactory`: known type ids =
[HLLSketchBuild, HLLSketchMerge, arrayOfDoublesSketch, cardinality, count,
doubleFirst, doubleLast, doubleMax, doubleMean, doubleMin, doubleSum, filtered,
floatFirst, floatLast, floatMax, floatMin, floatSum, histogram, hyperUnique,
javascript, longFirst, longLast, longMax, longMin, longSum,
quantilesDoublesSketch, quantilesDoublesSketchMerge, sketchBuild, sketchMerge,
stringFirst, stringFirstFold, stringLast, stringLastFold, thetaSketch, timeMax,
timeMin] (for POJO property 'aggregators')
at [Source:
(byte[])"{"container":{},"aggregators":[{"type":"longSum","name":"count","fieldName":"count","expression":null},{"type":"longSum","name":"data_bytes_from_dst_sum","fieldName":"data_bytes_from_dst_sum","expression":null},{"type":"longSum","name":"data_bytes_from_src_sum","fieldName":"data_bytes_from_src_sum","expression":null},{"type":"TimestampAggregatorFactory","name":"first_seen","fieldName":"first_seen","timeFormat":null},{"type":"TimestampAggregatorFactory","name":"last_seen","fieldName":"last_seen","[truncated
380 bytes]; line: 1, column: 328] (through reference chain:
org.apache.druid.segment.Metadata["aggregators"]->java.lang.Object[][3])
at
com.fasterxml.jackson.databind.exc.InvalidTypeIdException.from(InvalidTypeIdException.java:43)
~[jackson-databind-2.10.1.jar:2.10.1]
at
com.fasterxml.jackson.databind.DeserializationContext.invalidTypeIdException(DeserializationContext.java:1758)
~[jackson-databind-2.10.1.jar:2.10.1]
at
com.fasterxml.jackson.databind.DeserializationContext.handleUnknownTypeId(DeserializationContext.java:1265)
~[jackson-databind-2.10.1.jar:2.10.1]
at
com.fasterxml.jackson.databind.jsontype.impl.TypeDeserializerBase._handleUnknownTypeId(TypeDeserializerBase.java:290)
~[jackson-databind-2.10.1.jar:2.10.1]
at
com.fasterxml.jackson.databind.jsontype.impl.TypeDeserializerBase._findDeserializer(TypeDeserializerBase.java:162)
~[jackson-databind-2.10.1.jar:2.10.1]
at
com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer._deserializeTypedForId(AsPropertyTypeDeserializer.java:113)
~[jackson-databind-2.10.1.jar:2.10.1]
at
com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject(AsPropertyTypeDeserializer.java:97)
~[jackson-databind-2.10.1.jar:2.10.1]
at
com.fasterxml.jackson.databind.deser.AbstractDeserializer.deserializeWithType(AbstractDeserializer.java:254)
~[jackson-databind-2.10.1.jar:2.10.1]
at
com.fasterxml.jackson.databind.deser.std.ObjectArrayDeserializer.deserialize(ObjectArrayDeserializer.java:197)
~[jackson-databind-2.10.1.jar:2.10.1]
at
com.fasterxml.jackson.databind.deser.std.ObjectArrayDeserializer.deserialize(ObjectArrayDeserializer.java:21)
~[jackson-databind-2.10.1.jar:2.10.1]
at
com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:530)
~[jackson-databind-2.10.1.jar:2.10.1]
at
com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeWithErrorWrapping(BeanDeserializer.java:528)
~[jackson-databind-2.10.1.jar:2.10.1]
at
com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:417)
~[jackson-databind-2.10.1.jar:2.10.1]
at
com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1287)
~[jackson-databind-2.10.1.jar:2.10.1]
at
com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:326)
~[jackson-databind-2.10.1.jar:2.10.1]
at
com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:159)
~[jackson-databind-2.10.1.jar:2.10.1]
at
com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4202)
~[jackson-databind-2.10.1.jar:2.10.1]
at
com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3266)
~[jackson-databind-2.10.1.jar:2.10.1]
at
org.apache.druid.segment.IndexIO$V9IndexLoader.load(IndexIO.java:570)
[druid-processing-0.17.1.jar:0.17.1]
at org.apache.druid.segment.IndexIO.loadIndex(IndexIO.java:194)
[druid-processing-0.17.1.jar:0.17.1]
at org.apache.druid.segment.IndexIO.loadIndex(IndexIO.java:185)
[druid-processing-0.17.1.jar:0.17.1]
at
org.apache.druid.segment.realtime.appenderator.AppenderatorImpl.persistHydrant(AppenderatorImpl.java:1348)
[druid-server-0.17.1.jar:0.17.1]
at
org.apache.druid.segment.realtime.appenderator.AppenderatorImpl.access$100(AppenderatorImpl.java:103)
[druid-server-0.17.1.jar:0.17.1]
at
org.apache.druid.segment.realtime.appenderator.AppenderatorImpl$3.call(AppenderatorImpl.java:544)
[druid-server-0.17.1.jar:0.17.1]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[?:1.8.0_252]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[?:1.8.0_252]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[?:1.8.0_252]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_252]
```
reported via https://the-asf.slack.com/archives/CJ8D1JTB8/p1595442889073500
see
https://stackoverflow.com/c/imply/questions/1501/1502?noredirect=1#comment1456_1502
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
