[
https://issues.apache.org/jira/browse/SPARK-34658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-34658.
----------------------------------
Resolution: Invalid
> Kryo serializer not working with Spark SQL module
> -------------------------------------------------
>
> Key: SPARK-34658
> URL: https://issues.apache.org/jira/browse/SPARK-34658
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 3.0.1, 3.0.2, 3.1.1
> Reporter: Nafis Sadat
> Priority: Minor
>
> Hey all,
> I apologize if this has already been explored, but I keep getting this error
> whenever I have the Kryo registration required in my Spark app. The error is
> below; I have tried many a different types of combination to register this
> class but I just can't get anything to work, and so any help will be highly
> appreciated!
>
> {code:java}
> Caused by: com.esotericsoftware.kryo.KryoException:
> java.lang.IllegalArgumentException: Class is not registered:
> org.apache.spark.sql.execution.datasources.InMemoryFileIndex$Seriali
> zableBlockLocation[]
> Note: To register this class use:
> kryo.register(org.apache.spark.sql.execution.datasources.InMemoryFileIndex$SerializableBlockLocation[].class);
> Serialization trace:
> blockLocations
> (org.apache.spark.sql.execution.datasources.InMemoryFileIndex$SerializableFileStatus)
> array (scala.collection.mutable.ArrayBuffer)
> {code}
>
> I've tried all kinds of things to get these classes to be registered, but
> it's just not working for me..
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
