squito commented on a change in pull request #24264: [SPARK-27216][CORE]
Upgrade RoaringBitmap to 0.7.45 to fix Kryo unsafe ser/dser issue
URL: https://github.com/apache/spark/pull/24264#discussion_r271128777
##########
File path:
core/src/test/scala/org/apache/spark/serializer/KryoSerializerSuite.scala
##########
@@ -565,6 +565,20 @@ class KryoSerializerAutoResetDisabledSuite extends
SparkFunSuite with SharedSpar
assert(serInstance.deserialize[Any](serObj) === (obj))
assert(serInstance.deserialize[Any](byteBuffer) === (obj))
}
+
+ test("SPARK-27216: Upgrade RoaringBitmap to 0.7.45 to fix Kryo unsafe
ser/dser issue") {
+ val expected = new RoaringBitmap()
+ expected.add(1787)
+
+ conf.set(KRYO_USE_UNSAFE, true)
Review comment:
I dont' think this is what @attilapiros meant. I think he meant to (a)
leave `UnsafeKryoSerializerSuite` as it was before and (b) add a new test which
just tested serialization of `RoaringBitmap` with kryo in this class, but
didn't tweak confs at all. That way you'd get a test for serializing with
kryo.unsafe=false (here in `KryoSerializerSuite`) and you'd also automatically
get a test with kryo.unsafe=true (inherited in `UnsafeKryoSerializerSuite`).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
