Github user sounakr commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2417#discussion_r201241294
--- Diff:
core/src/main/java/org/apache/carbondata/core/datastore/chunk/store/ColumnPageWrapper.java
---
@@ -53,11 +60,75 @@ public int fillVector(int[] filteredRowId,
ColumnVectorInfo[] vectorInfo, int ch
throw new UnsupportedOperationException("internal error");
}
- @Override
- public byte[] getChunkData(int rowId) {
- return columnPage.getBytes(rowId);
+ @Override public byte[] getChunkData(int rowId) {
+ ColumnType columnType = columnPage.getColumnSpec().getColumnType();
+ DataType srcDataType = columnPage.getColumnSpec().getSchemaDataType();
+ DataType targetDataType = columnPage.getDataType();
+ if (columnPage.getNullBits().get(rowId)) {
+ // if this row is null, return default null represent in byte array
+ return CarbonCommonConstants.MEMBER_DEFAULT_VAL_ARRAY;
+ }
+ if ((columnType == ColumnType.COMPLEX_PRIMITIVE) &&
this.isAdaptiveComplexPrimitive()) {
+ if (srcDataType == DataTypes.DOUBLE || srcDataType ==
DataTypes.FLOAT) {
+ double doubleData = columnPage.getDouble(rowId);
+ if (srcDataType == DataTypes.FLOAT) {
+ float out = (float) doubleData;
+ return ByteUtil.toBytes(out);
+ } else {
+ return ByteUtil.toBytes(doubleData);
+ }
+ } else if (DataTypes.isDecimal(srcDataType)) {
+ throw new RuntimeException("unsupported type: " + srcDataType);
+ } else if ((srcDataType == DataTypes.BYTE) ||
+ (srcDataType == DataTypes.BOOLEAN) ||
+ (srcDataType == DataTypes.SHORT) ||
+ (srcDataType == DataTypes.SHORT_INT) ||
+ (srcDataType == DataTypes.INT) ||
+ (srcDataType == DataTypes.LONG) ||
+ (srcDataType == DataTypes.TIMESTAMP)) {
+ long longData = columnPage.getLong(rowId);
--- End diff --
There are altogether three level of Column Page Abstraction.
ColumnPageWrapper
LazyColumnPage
Safe | unsafe | variable | fixed. Column Page.
The below layer after return data based on DataType and the value is
converted and given back to LazyColumnPage. As the Data is Adaptive or Adaptive
Delta Encoded, LazyColumnPage Decode the data after negating with Max value in
page statistics. This Max value is either long, double or decimal. So Lazy has
to get the data in long, double or decimal format to make it comparable with
Max long value. Therefore in Lazy getLong, getDouble and getDecimal are
implemented in order to decode the data (negating with max) and subsequently
the data is returned as Long. So in the case of Integral codec the returned
value will be long, in floating adaptive it will be double and for decimal it
will be returned as decimal.
So here getLong is correct and the data we will be getting in long format
Later they are reconverted back to respective datatypes.
---
