[
https://issues.apache.org/jira/browse/SPARK-45081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun closed SPARK-45081.
---------------------------------
> Encoders.bean does no longer work with read-only properties
> -----------------------------------------------------------
>
> Key: SPARK-45081
> URL: https://issues.apache.org/jira/browse/SPARK-45081
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 3.4.1
> Reporter: Giambattista Bloisi
> Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.2, 3.5.1, 4.0.0
>
>
> Since Spark 3.4.x an exception is thrown when Encoders.bean is called
> providing a bean having read-only properties, such as:
>
> {code:java}
> public static class ReadOnlyPropertyBean implements Serializable {
> public boolean isEmpty() {
> return true;
> }
> } {code}
>
>
> Encoders.bean(ReadOnlyPropertyBean.class) will throw:
> {code:java}
> java.util.NoSuchElementException: None.get
> at scala.None$.get(Option.scala:529)
> at scala.None$.get(Option.scala:527)
> at
> org.apache.spark.sql.catalyst.ScalaReflection$.$anonfun$deserializerFor$8(ScalaReflection.scala:359)
> at
> scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:286)
> at
> scala.collection.IndexedSeqOptimized.foreach(IndexedSeqOptimized.scala:36)
> at
> scala.collection.IndexedSeqOptimized.foreach$(IndexedSeqOptimized.scala:33)
> at
> scala.collection.mutable.WrappedArray.foreach(WrappedArray.scala:38)
> at scala.collection.TraversableLike.map(TraversableLike.scala:286)
> at scala.collection.TraversableLike.map$(TraversableLike.scala:279)
> at scala.collection.AbstractTraversable.map(Traversable.scala:108)
> at
> org.apache.spark.sql.catalyst.ScalaReflection$.deserializerFor(ScalaReflection.scala:348)
> at
> org.apache.spark.sql.catalyst.ScalaReflection$.deserializerFor(ScalaReflection.scala:183)
> at
> org.apache.spark.sql.catalyst.encoders.ExpressionEncoder$.apply(ExpressionEncoder.scala:56)
> at
> org.apache.spark.sql.catalyst.encoders.ExpressionEncoder$.javaBean(ExpressionEncoder.scala:62)
> at org.apache.spark.sql.Encoders$.bean(Encoders.scala:179)
> at org.apache.spark.sql.Encoders.bean(Encoders.scala) {code}
> This problem is described also in [link Encoders.bean doesn't work anymore on
> a Java POJO, with Spark
> 3.4.0|https://stackoverflow.com/questions/76036349/encoders-bean-doesnt-work-anymore-on-a-java-pojo-with-spark-3-4-0]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
