Github user ravipesala commented on a diff in the pull request:
https://github.com/apache/incubator-carbondata/pull/639#discussion_r105883239
--- Diff:
integration/spark-common/src/main/scala/org/apache/carbondata/spark/util/DataTypeConverterUtil.scala
---
@@ -56,7 +56,9 @@ object DataTypeConverterUtil {
case "decimaltype" => DataType.DECIMAL
case "timestamptype" => DataType.TIMESTAMP
case "datetype" => DataType.DATE
- case "arraytype" => DataType.ARRAY
+ case "arraytype" | "arraytype(stringtype,true)" |
"arraytype(inttype,true)" |
--- End diff --
I think it would be the same case with structtype as well. Please simplify
as follows.
```
def convertToCarbonTypeForSpark2(dataType: String): DataType = {
dataType.toLowerCase match {
case "stringtype" => DataType.STRING
case "inttype" => DataType.INT
case "integertype" => DataType.INT
case "tinyinttype" => DataType.SHORT
case "shorttype" => DataType.SHORT
case "longtype" => DataType.LONG
case "biginttype" => DataType.LONG
case "numerictype" => DataType.DOUBLE
case "doubletype" => DataType.DOUBLE
case "floattype" => DataType.DOUBLE
case "decimaltype" => DataType.DECIMAL
case "timestamptype" => DataType.TIMESTAMP
case "datetype" => DataType.DATE
case others =>
if (others != null && others.startsWith("arraytype")) {
DataType.ARRAY
} else if (others != null && others.startsWith("structtype")) {
DataType.STRUCT
} else {
sys.error(s"Unsupported data type: $dataType")
}
}
}
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
