[ https://issues.apache.org/jira/browse/SPARK-35267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ganesh Chand updated SPARK-35267: --------------------------------- Description: {code:java} // code placeholder object Util { def toStructType[T](implicit typeTags: ScalaReflection.universe.TypeTag[T]): StructType = ScalaReflection.schemaFor[T].dataType.asInstanceOf[StructType] } case class MyTable(val a: Int, val b: String) // The following test fails because schema returned using reflection sets nullable=false for integer column. The test passes if you set it to false it must "return a Spark Schema of type StructType for a case class" in { val schemaFromCaseClass: StructType = Util.toStructType[MyTable] val expectedSchema = new StructType().add(StructField("a", IntegerType, true)).add(StructField("b", StringType, true)) schemaFromCaseClass.size mustBe 2 schemaFromCaseClass mustBe expectedSchema } {code} was: {code:java} // code placeholder object Util { def toStructType[T](implicit typeTags: ScalaReflection.universe.TypeTag[T]): StructType = ScalaReflection.schemaFor[T].dataType.asInstanceOf[StructType] } case class MyTable(val a: Int, val b: String) // The following test fails because schema returned using reflection sets nullable=false for integer column. The test passes if you set it to false it must "return a Spark Schema of type StructType for a case class" in { val schemaFromCaseClass: StructType = Util.toStructType[MyTable] val expectedSchema = new StructType().add(StructField("a", IntegerType, true)).add(StructField("b", StringType, true)) schemaFromCaseClass.size mustBe 2 schemaFromCaseClass mustBe expectedSchema } {code} > nullable field is set to false for integer type when using reflection to get > StructType for a case class > -------------------------------------------------------------------------------------------------------- > > Key: SPARK-35267 > URL: https://issues.apache.org/jira/browse/SPARK-35267 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 3.1.1 > Environment: Scala version: 2.12.13 > sparkVersion = "3.1.1" > Reporter: Ganesh Chand > Priority: Major > > {code:java} > // code placeholder > object Util { > def toStructType[T](implicit typeTags: > ScalaReflection.universe.TypeTag[T]): StructType = > ScalaReflection.schemaFor[T].dataType.asInstanceOf[StructType] > } > case class MyTable(val a: Int, val b: String) > // The following test fails because schema returned using reflection sets > nullable=false for integer column. The test passes if you set it to false > it must "return a Spark Schema of type StructType for a case class" in { > val schemaFromCaseClass: StructType = Util.toStructType[MyTable] > val expectedSchema = new StructType().add(StructField("a", IntegerType, > true)).add(StructField("b", StringType, true)) > schemaFromCaseClass.size mustBe 2 > schemaFromCaseClass mustBe expectedSchema > } > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org