singhpk234 edited a comment on issue #4288: URL: https://github.com/apache/iceberg/issues/4288#issuecomment-1062844207
yup I think you are right as per this [SparkCodePointer](https://github.com/apache/spark/blob/master/core%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fspark%2Fserializer%2FKryoSerializer.scala#L398-L411) ```scala override def deserialize[T: ClassTag](bytes: ByteBuffer): T = { val kryo = borrowKryo() try { if (bytes.hasArray) { input.setBuffer(bytes.array(), bytes.arrayOffset() + bytes.position(), bytes.remaining()) } else { input.setBuffer(new Array[Byte](4096)) input.setInputStream(new ByteBufferInputStream(bytes)) } kryo.readClassAndObject(input).asInstanceOf[T] } finally { releaseKryo(kryo) } } ``` kryo is using it's own readClassAndObject rather than readObject, whereas JavaSerializer uses readObject ([CodePointer1](https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/serializer/JavaSerializer.scala#L123), [CodePointer2](https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/serializer/JavaSerializer.scala#L87)). I beleive this could explain the NPE -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
