Abhilash Srimat Tirumala Pallerlamudi created SPARK-11827:
-------------------------------------------------------------
Summary: Support java.math.BigInteger in Type-Inference utilities
for POJOs
Key: SPARK-11827
URL: https://issues.apache.org/jira/browse/SPARK-11827
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 1.5.1
Reporter: Abhilash Srimat Tirumala Pallerlamudi
Priority: Minor
I get the below exception when creating DataFrame using RDD of JavaBean having
a property of type java.math.BigInteger
scala.MatchError: class java.math.BigInteger (of class java.lang.Class)
at
org.apache.spark.sql.SQLContext$$anonfun$getSchema$1.apply(SQLContext.scala:1182)
at
org.apache.spark.sql.SQLContext$$anonfun$getSchema$1.apply(SQLContext.scala:1181)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
at
scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:108)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
at scala.collection.mutable.ArrayOps$ofRef.map(ArrayOps.scala:108)
at org.apache.spark.sql.SQLContext.getSchema(SQLContext.scala:1181)
at org.apache.spark.sql.SQLContext.createDataFrame(SQLContext.scala:419)
at org.apache.spark.sql.SQLContext.createDataFrame(SQLContext.scala:447)
I don't see the support for java.math.BigInteger in
org.apache.spark.sql.catalyst.JavaTypeInference.scala
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
