SaschaC created CALCITE-2256:
--------------------------------
Summary: Exception in Druid Adapter: JsonSegmentMetadata, numRows
field out of range
Key: CALCITE-2256
URL: https://issues.apache.org/jira/browse/CALCITE-2256
Project: Calcite
Issue Type: Bug
Components: druid
Affects Versions: 1.16.0
Reporter: SaschaC
Assignee: Julian Hyde
When trying to connect to a Druid datasource, Calcite pulls metadata from Druid
as a first action. As Druid is a database for big-data, it is not uncommon that
a datasource would contain billions of records. The metadata about a datasource
returns a records count, namely the total number of records residing in that
datasource. This is returned in a property named "numRows".
The Calcite Druid Adapter tries to read the numRows field (that Druid returns
within the response Json document of a metadata request) into an integer field
which overflows for record counts bigger than 4 billion and triggers the below
exception
{code}
Caused by: com.fasterxml.jackson.databind.JsonMappingException: Numeric value
(13452491153) out of range of int
at [Source: (sun.net.www.protocol.http.HttpURLConnection$HttpInputStream);
line: 565, column: 26]
at [Source: (sun.net.www.protocol.http.HttpURLConnection$HttpInputStream);
line: 565, column: 15] (through reference chain:
java.util.ArrayList[0]->org.apache.calcite.adapter.druid.DruidConnectionImpl$JsonSegmentMetadata["numRows"])
{code}
In class org.apache.calcite.adapter.druid,*DruidConnectionImpl*, inner class
JsonSegmentMetadata has integer fields for reading in Druid metadata like size
or numRows which aught to be of type *long* so that they can hold larger
numbers:
{code}
private static class JsonSegmentMetadata {
public String id;
public List<String> intervals;
public Map<String, JsonColumn> columns;
*public int size;*
*public int numRows;*
public Map<String, JsonAggregator> aggregators;
}
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
