Hi everybody! Me again. I'm developing Spark API for Kotlin and found interesting behaviour:
As you know, Kotlin has concept of nullable types which I'm using in my implementation of serializers/deserializers (slightly changed ScalaReflection to be more concrete). But in one case it doesn't work for me: when list contains nullable type It outputs error like this: Caused by: java.lang.NullPointerException: Null value appeared in non-nullable field: top level Product or row object If the schema is inferred from a Scala tuple/case class, or a Java bean, please try to use scala.Option[_] or other nullable types (e.g. java.lang.Integer instead of int/scala.Int). What is even more interesting that if I disable "spark.sql.codegen.wholeStage" than program works as intended. Any ideas on this? Thanks, Pasha --------------------------------------------------------------------- To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
