[GitHub] carbondata pull request #1934: [CARBONDATA-2133] Fixed Exception displays af...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/carbondata/pull/1934


---

[GitHub] carbondata pull request #1934: [CARBONDATA-2133] Fixed Exception displays af...

[anubhav100]

Github user anubhav100 commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/1934#discussion_r168088520
  
--- Diff: 
integration/spark2/src/test/scala/org/apache/spark/carbondata/restructure/AlterTableValidationTestCase.scala
 ---
@@ -544,6 +546,13 @@ class AlterTableValidationTestCase extends 
Spark2QueryTest with BeforeAndAfterAl
 sql("drop table if exists restructure1")
 sql("drop table if exists restructure")
   }
+test("test alter command for boolean data type with correct default 
measure value") {
+  sql("create table testalterwithboolean(id int,name string) stored by 
'carbondata' ")
+  sql("insert into testalterwithboolean values(1,'anubhav')  ")
+  sql(
+"alter table testalterwithboolean add columns(booleanfield boolean) 
tblproperties('default.value.booleanfield'='true')")
+  checkAnswer(sql("select * from 
testalterwithboolean"),Seq(Row(1,"anubhav",true)))
+}
--- End diff --

done


---

[GitHub] carbondata pull request #1934: [CARBONDATA-2133] Fixed Exception displays af...

[manishgupta88]

Github user manishgupta88 commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/1934#discussion_r168083518
  
--- Diff: 
integration/spark2/src/test/scala/org/apache/spark/carbondata/restructure/AlterTableValidationTestCase.scala
 ---
@@ -544,6 +546,13 @@ class AlterTableValidationTestCase extends 
Spark2QueryTest with BeforeAndAfterAl
 sql("drop table if exists restructure1")
 sql("drop table if exists restructure")
   }
+test("test alter command for boolean data type with correct default 
measure value") {
+  sql("create table testalterwithboolean(id int,name string) stored by 
'carbondata' ")
+  sql("insert into testalterwithboolean values(1,'anubhav')  ")
+  sql(
+"alter table testalterwithboolean add columns(booleanfield boolean) 
tblproperties('default.value.booleanfield'='true')")
+  checkAnswer(sql("select * from 
testalterwithboolean"),Seq(Row(1,"anubhav",true)))
+}
--- End diff --

Please add one more test case for boolean type where the default value is 
not provided


---

[GitHub] carbondata pull request #1934: [CARBONDATA-2133] Fixed Exception displays af...

[manishgupta88]

Github user manishgupta88 commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/1934#discussion_r168083426
  
--- Diff: 
core/src/main/java/org/apache/carbondata/core/util/DataTypeUtil.java ---
@@ -720,21 +721,22 @@ private static String parseStringToBigDecimal(String 
value, ColumnSchema columnS
 return null;
   }
 
+  public static DataTypeConverter getDataTypeConverter() {
+if (converter == null) {
+  converter = new DataTypeConverterImpl();
+}
+return converter;
+  }
+
--- End diff --

This change does not belong to this PR...the code is already present in the 
same class...kindly remove all the changes that does not belong to this PR


---

[GitHub] carbondata pull request #1934: [CARBONDATA-2133] Fixed Exception displays af...

[anubhav100]

GitHub user anubhav100 opened a pull request:

https://github.com/apache/carbondata/pull/1934

[CARBONDATA-2133] Fixed Exception displays after performing select query on 
newly added Boolean Type




**Problem** : In Restructure util and  
RestructureBasedVectorResultCollector to get the default value of a measure 
type the case for boolean data type was missing,and in DataTypeUtil to store 
default value in bytes case of boolean data type was missing

**Solution**:Add the Required Cases

**Complete Description**:

**Classes changed**:

1.**RestructureBasedVectorResultCollector**:

**in the below method case for boolean data type was missing**

private void fillDataForNonExistingMeasures() {
for (int i = 0; i < 
tableBlockExecutionInfos.getActualQueryMeasures().length; i++) {
  if (!measureInfo.getMeasureExists()[i]) {
int queryOrder = 
tableBlockExecutionInfos.getActualQueryMeasures()[i].getQueryOrder();
CarbonMeasure measure = 
tableBlockExecutionInfos.getActualQueryMeasures()[i].getMeasure();
ColumnVectorInfo columnVectorInfo = allColumnInfo[queryOrder];
CarbonColumnVector vector = columnVectorInfo.vector;
Object defaultValue = measureDefaultValues[i];
if (null == defaultValue) {
  vector.putNulls(columnVectorInfo.vectorOffset, 
columnVectorInfo.size);
} else {
  DataType dataType = measureInfo.getMeasureDataTypes()[i];
  if (dataType == DataTypes.SHORT) {
vector.putShorts(columnVectorInfo.vectorOffset, 
columnVectorInfo.size,
(short) defaultValue);
  } else if (dataType == DataTypes.INT) {
vector
.putInts(columnVectorInfo.vectorOffset, 
columnVectorInfo.size, (int) defaultValue);
  } else if (dataType == DataTypes.LONG) {
vector.putLongs(columnVectorInfo.vectorOffset, 
columnVectorInfo.size,
(long) defaultValue);
  } else if (DataTypes.isDecimal(dataType)) {
vector.putDecimals(columnVectorInfo.vectorOffset, 
columnVectorInfo.size,
((Decimal) defaultValue).toJavaBigDecimal(), 
measure.getPrecision());
  } else {
vector.putDoubles(columnVectorInfo.vectorOffset, 
columnVectorInfo.size,
(double) defaultValue);
  }
}
  }
}
  }

2.**RestructureUtil**:

**in below methods getMeasureDefaultValue,getMeasureDefaultValueByType the 
if statement  for boolean data type was missing**

public static Object getMeasureDefaultValue(ColumnSchema columnSchema, 
byte[] defaultValue) {
Object measureDefaultValue = null;
if (!isDefaultValueNull(defaultValue)) {
  String value;
  DataType dataType = columnSchema.getDataType();
  if (dataType == DataTypes.SHORT) {
value = new String(defaultValue, 
Charset.forName(CarbonCommonConstants.DEFAULT_CHARSET));
measureDefaultValue = Short.valueOf(value);
  } else if (dataType == DataTypes.LONG) {
value = new String(defaultValue, 
Charset.forName(CarbonCommonConstants.DEFAULT_CHARSET));
measureDefaultValue = Long.parseLong(value);
  } else if (dataType == DataTypes.INT) {
value = new String(defaultValue, 
Charset.forName(CarbonCommonConstants.DEFAULT_CHARSET));
measureDefaultValue = Integer.parseInt(value);
  } else if (DataTypes.isDecimal(dataType)) {
BigDecimal decimal = DataTypeUtil.byteToBigDecimal(defaultValue);
if (columnSchema.getScale() > decimal.scale()) {
  decimal = decimal.setScale(columnSchema.getScale(), 
RoundingMode.HALF_UP);
}
measureDefaultValue = decimal;
  } else {
value = new String(defaultValue, 
Charset.forName(CarbonCommonConstants.DEFAULT_CHARSET));
Double parsedValue = Double.valueOf(value);
if (!Double.isInfinite(parsedValue) && !Double.isNaN(parsedValue)) {
  measureDefaultValue = parsedValue;
}
  }
}
return measureDefaultValue;
  }

3.**DatatypeUtil**: To store default value on bytes case of boolean data 
type was missing

public static byte[] convertDataToBytesBasedOnDataType(String data, 
ColumnSchema columnSchema) {
if (null == data) {
  return null;
} else if (CarbonCommonConstants.MEMBER_DEFAULT_VAL.equals(data)) {
  LOGGER.error("Default value should not be carbon specific null value 
: " + data);
  return null;
}
try {
  long parsedIntVal = 0;
  DataType dataType = columnSchema.getDataType();
  if (dataType == DataTypes.INT) {
parsedIntVal = (long)