Giambattista Bloisi created SPARK-45081:
-------------------------------------------
Summary: Encoders.bean does no longer work with read-only
properties
Key: SPARK-45081
URL: https://issues.apache.org/jira/browse/SPARK-45081
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 3.4.1
Reporter: Giambattista Bloisi
Since Spark 3.4.x an exception is thrown when Encoders.bean is called providing
a bean having read-only properties, such as:
{code:java}
public static class ReadOnlyPropertyBean implements Serializable {
public boolean isEmpty() {
return true;
}
} {code}
Encoders.bean(ReadOnlyPropertyBean.class) will throw:
{code:java}
java.util.NoSuchElementException: None.get
at scala.None$.get(Option.scala:529)
at scala.None$.get(Option.scala:527)
at
org.apache.spark.sql.catalyst.ScalaReflection$.$anonfun$deserializerFor$8(ScalaReflection.scala:359)
at
scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:286)
at
scala.collection.IndexedSeqOptimized.foreach(IndexedSeqOptimized.scala:36)
at
scala.collection.IndexedSeqOptimized.foreach$(IndexedSeqOptimized.scala:33)
at scala.collection.mutable.WrappedArray.foreach(WrappedArray.scala:38)
at scala.collection.TraversableLike.map(TraversableLike.scala:286)
at scala.collection.TraversableLike.map$(TraversableLike.scala:279)
at scala.collection.AbstractTraversable.map(Traversable.scala:108)
at
org.apache.spark.sql.catalyst.ScalaReflection$.deserializerFor(ScalaReflection.scala:348)
at
org.apache.spark.sql.catalyst.ScalaReflection$.deserializerFor(ScalaReflection.scala:183)
at
org.apache.spark.sql.catalyst.encoders.ExpressionEncoder$.apply(ExpressionEncoder.scala:56)
at
org.apache.spark.sql.catalyst.encoders.ExpressionEncoder$.javaBean(ExpressionEncoder.scala:62)
at org.apache.spark.sql.Encoders$.bean(Encoders.scala:179)
at org.apache.spark.sql.Encoders.bean(Encoders.scala) {code}
This problem is described also in [link Encoders.bean doesn't work anymore on a
Java POJO, with Spark
3.4.0|https://stackoverflow.com/questions/76036349/encoders-bean-doesnt-work-anymore-on-a-java-pojo-with-spark-3-4-0]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
