Re: Why is DecimalType separate from DataType ?

2015-01-30 Thread Michael Armbrust 
You are grabbing the singleton, not the class.  You need to specify the
precision (i.e. DecimalType.Unlimited or DecimalType(precision, scale))

On Fri, Jan 30, 2015 at 2:23 PM, Manoj Samel 
wrote:

> Spark 1.2
>
> While building schemaRDD using StructType
>
> xxx = new StructField("credit_amount", DecimalType, true) gives error
> "type mismatch; found :
> org.apache.spark.sql.catalyst.types.DecimalType.type required:
> org.apache.spark.sql.catalyst.types.DataType"
>
> From
> https://spark.apache.org/docs/1.2.0/api/scala/index.html#org.apache.spark.sql.package,
> it seems DecimalType = sql.catalyst.types.DecimalType is separate from
> DataType = sql.catalyst.types.DataType
>
> Not sure why that is the case? How does one uses Decimal and other types
> in StructField?
>
> Thanks,
>
>
>
>

Why is DecimalType separate from DataType ?

2015-01-30 Thread Manoj Samel 
Spark 1.2

While building schemaRDD using StructType

xxx = new StructField("credit_amount", DecimalType, true) gives error "type
mismatch; found : org.apache.spark.sql.catalyst.types.DecimalType.type
required: org.apache.spark.sql.catalyst.types.DataType"

From
https://spark.apache.org/docs/1.2.0/api/scala/index.html#org.apache.spark.sql.package,
it seems DecimalType = sql.catalyst.types.DecimalType is separate from
DataType = sql.catalyst.types.DataType

Not sure why that is the case? How does one uses Decimal and other types in
StructField?

Thanks,