[ https://issues.apache.org/jira/browse/CALCITE-2483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16591105#comment-16591105 ]
Hongze Zhang commented on CALCITE-2483: --------------------------------------- [~julianhyde] Thanks for your reply, I have changed the commit comment as you suggested. {quote}Is a test case feasible? {quote} IMHO, feasible, but not necessary enough. Since the exception came inside the [deserialization|https://github.com/apache/calcite/blob/master/druid/src/main/java/org/apache/calcite/adapter/druid/DruidConnectionImpl.java#L578], any test for this issue should start with mocking a original JSON text including a large rowNumber, then pass it to Druid adapter, and assert no deserialization error thrown. A test like this is mostly like testing the Jackson lib itself, which should be already covered by Jackson source. What do you think? {quote}Have you checked that usages of those fields are also long values? {quote} Yes, I had checked the usage before submitting the PR, I am afraid that the two changed fields are not used in Calcite code currently. > Druid adapter, when querying Druid segment metadata, throws when row number > is larger than Integer.MAX_VALUE > ------------------------------------------------------------------------------------------------------------ > > Key: CALCITE-2483 > URL: https://issues.apache.org/jira/browse/CALCITE-2483 > Project: Calcite > Issue Type: Bug > Components: druid > Affects Versions: 1.17.0 > Reporter: Hongze Zhang > Assignee: Julian Hyde > Priority: Major > > Calcite sends "merge" segment metadata query to Druid to fetch table schema, > in the response if the value of "numRows" is beyound Integer.MAX_VALUE, it > will fail JSON deserialization. > > {code:java} > com.google.common.util.concurrent.UncheckedExecutionException: > java.lang.RuntimeException: > com.fasterxml.jackson.databind.JsonMappingException: Numeric value > (19629299194) out of range owf int > at [Source: (sun.net.www.protocol.http.HttpURLConnection$HttpInputStream); > line: 241, column: 26] > at [Source: (sun.net.www.protocol.http.HttpURLConnection$HttpInputStream); > line: 241, column: 15] (through reference chain: > java.util.ArrayList[0]->org.apache.calcite.adapter.druid.DruidConnectionImpl$JsonSegmentMetadata["numRows"]) > at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2207) > at com.google.common.cache.LocalCache.get(LocalCache.java:3953) > at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3957) > at > com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4875) > at > com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4881) > at > com.google.common.cache.LocalCache$LocalLoadingCache.apply(LocalCache.java:4899) > at com.google.common.collect.Maps$AsMapView.get(Maps.java:805) > at > org.apache.calcite.schema.impl.AbstractSchema.getTable(AbstractSchema.java:89) > at > org.apache.calcite.jdbc.CachingCalciteSchema.getImplicitTable(CachingCalciteSchema.java:137) > at org.apache.calcite.jdbc.CalciteSchema.getTable(CalciteSchema.java:288) > at > org.apache.calcite.jdbc.CalciteMetaImpl.lambda$tables$8(CalciteMetaImpl.java:383) > at > org.apache.calcite.linq4j.EnumerableDefaults$11$1.current(EnumerableDefaults.java:1871) > at > org.apache.calcite.linq4j.Linq4j$CompositeEnumerable$1.current(Linq4j.java:498) > at > org.apache.calcite.linq4j.EnumerableDefaults$24.moveNext(EnumerableDefaults.java:2767) > at > org.apache.calcite.linq4j.EnumerableDefaults$13$1.moveNext(EnumerableDefaults.java:1948) > at > org.apache.calcite.linq4j.EnumerableDefaults$24.moveNext(EnumerableDefaults.java:2766) > at org.apache.calcite.linq4j.Linq4j$EnumeratorIterator.next(Linq4j.java:685) > at org.apache.calcite.avatica.util.IteratorCursor.next(IteratorCursor.java:46) > at org.apache.calcite.avatica.AvaticaResultSet.next(AvaticaResultSet.java:217) > at sqlline.IncrementalRows.hasNext(IncrementalRows.java:65) > at sqlline.TableOutputFormat.print(TableOutputFormat.java:33) > at sqlline.SqlLine.print(SqlLine.java:1648) > at sqlline.Commands.metadata(Commands.java:199) > at sqlline.Commands.tables(Commands.java:332) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:38) > at sqlline.SqlLine.dispatch(SqlLine.java:791) > at sqlline.SqlLine.begin(SqlLine.java:668) > at sqlline.SqlLine.start(SqlLine.java:373) > at sqlline.SqlLine.main(SqlLine.java:265) > Caused by: java.lang.RuntimeException: > com.fasterxml.jackson.databind.JsonMappingException: Numeric value > (19629299194) out of range of int > at [Source: (sun.net.www.protocol.http.HttpURLConnection$HttpInputStream); > line: 241, column: 26] > at [Source: (sun.net.www.protocol.http.HttpURLConnection$HttpInputStream); > line: 241, column: 15] (through reference chain: > java.util.ArrayList[0]->org.apache.calcite.adapter.druid.DruidConnectionImpl$JsonSegmentMetadata["numRows"]) > at > org.apache.calcite.adapter.druid.DruidConnectionImpl.metadata(DruidConnectionImpl.java:617) > at org.apache.calcite.adapter.druid.DruidSchema.table(DruidSchema.java:85) > at > org.apache.calcite.adapter.druid.DruidSchema.lambda$getTableMap$0(DruidSchema.java:74) > at > com.google.common.cache.CacheLoader$FunctionToCacheLoader.load(CacheLoader.java:149) > at > com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3542) > at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2323) > at > com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2286) > at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2201) > ... 32 more > Caused by: com.fasterxml.jackson.databind.JsonMappingException: Numeric value > (19629299194) out of range of int > at [Source: (sun.net.www.protocol.http.HttpURLConnection$HttpInputStream); > line: 241, column: 26] > at [Source: (sun.net.www.protocol.http.HttpURLConnection$HttpInputStream); > line: 241, column: 15] (through reference chain: > java.util.ArrayList[0]->org.apache.calcite.adapter.druid.DruidConnectionImpl$JsonSegmentMetadata["numRows"]) > at > com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:391) > at > com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:351) > at > com.fasterxml.jackson.databind.deser.BeanDeserializerBase.wrapAndThrow(BeanDeserializerBase.java:1704) > at > com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:290) > at > com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:151) > at > com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:286) > at > com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:245) > at > com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:27) > at > com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4001) > at > com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3072) > at > org.apache.calcite.adapter.druid.DruidConnectionImpl.metadata(DruidConnectionImpl.java:578) > ... 39 more > Caused by: com.fasterxml.jackson.core.JsonParseException: Numeric value > (19629299194) out of range of int > at [Source: (sun.net.www.protocol.http.HttpURLConnection$HttpInputStream); > line: 241, column: 26] > at com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:1804) > at > com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:663) > at > com.fasterxml.jackson.core.base.ParserBase.convertNumberToInt(ParserBase.java:869) > at > com.fasterxml.jackson.core.base.ParserBase._parseIntValue(ParserBase.java:801) > at com.fasterxml.jackson.core.base.ParserBase.getIntValue(ParserBase.java:645) > at > com.fasterxml.jackson.databind.deser.std.NumberDeserializers$IntegerDeserializer.deserialize(NumberDeserializers.java:472) > at > com.fasterxml.jackson.databind.deser.std.NumberDeserializers$IntegerDeserializer.deserialize(NumberDeserializers.java:452) > at > com.fasterxml.jackson.databind.deser.impl.FieldProperty.deserializeAndSet(FieldProperty.java:136) > at > com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:288) > ... 46 more > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)