Hi there,
I'm storing RoaringBitmaps in Ignite and have encountered an odd serialization
issue. Please forgive the samples below being in Clojure, I've written a small
wrapper around most Ignite APIs that I can use. I think you can catch the gist
of what I'm doing, I hope :)
(let [inst (i/instance "attribute_bitmaps")
bmp (doto (RoaringBitmap.)
(.add 9999999))
srz (fn [it]
(let [b (ByteArrayOutputStream.)]
(with-open [o (DataOutputStream. b)]
(.serialize it o))
(.toByteArray b)))]
(doto inst
(i/clear)
(i/put "test" bmp))
(byte-streams/print-bytes (srz bmp))
(byte-streams/print-bytes (srz (i/get inst "test"))))
Here I'm just creating a bitmap and storing it in a cache. I'm then just
printing the bytes of the thing I stored, as well as what it looks like coming
back out of Ignite.
I get the following warning in the logs:
10:16:45.730 [djura.local ~ nREPL-worker-3 ~
o.a.i.internal.binary.BinaryContext] Class "org.roaringbitmap.RoaringBitmap"
cannot be serialized using BinaryMarshaller because it either implements
Externalizable interface or have writeObject/readObject methods.
OptimizedMarshaller will be used instead and class instances will be
deserialized on the server. Please ensure that all nodes have this class in
classpath. To enable binary serialization either implement Binarylizable
interface or set explicit serializer using
BinaryTypeConfiguration.setSerializer() method.
And really strangely, I get the same number of bytes back but some of them at
the end have been zero'd out (first one is correct, second one went through
Ignite):
3A 30 00 00 01 00 00 00 98 00 00 00 10 00 00 00 .0..............
7F 96 ..
3A 30 00 00 01 00 00 00 98 00 00 00 10 00 00 00 .0..............
00 00 ..
The resulting object is still a valid RoaringBitmap, except all the values in
the bitmap are wrong!
I'm guessing from this and the logs, that OptimizedMarshaller is being used
instead of BinaryMarshaller, and OptimizedMarshaller is not copying the
internal fields of the class correctly.
Would the recommended approach here be to create a custom class that extends
RoaringBitmap and implements Binarylizable? I'm not sure if Binarylizable is a
suitable approach for this situation where I don't control the source for the
class in question. I have no knowledge of the internal fields of this class and
really just want to ensure it survives the roundtrip through Ignite by using
its own internal serialization mechanism.
Luke.
