I did just manage to use BinarySerializer and registered it as a custom
serializer, it seems to work pending some more tests. Again, curious to hear if
this approach is appropriate for the use case. Here's the Clojure version:
(defn roaring-serializer []
(reify BinarySerializer
(writeBinary [this o binaryWriter]
; o contains our object.
(let [byte-stream (ByteArrayOutputStream.)]
(with-open [out-stream (DataOutputStream. byte-stream)]
(.serialize ^RoaringBitmap o out-stream))
(.writeByteArray binaryWriter "val" (.toByteArray byte-stream))))
(readBinary [this o binaryReader]
; o contains an empty object
(let [raw-bytes (.readByteArray binaryReader "val")
byte-stream (ByteArrayInputStream. raw-bytes)]
(with-open [in-stream (DataInputStream. byte-stream)]
(.deserialize ^RoaringBitmap o in-stream))))))
> On Mar 15, 2017, at 10:34 AM, Luke Burton <[email protected]> wrote:
>
>
> Hi there,
>
> I'm storing RoaringBitmaps in Ignite and have encountered an odd
> serialization issue. Please forgive the samples below being in Clojure, I've
> written a small wrapper around most Ignite APIs that I can use. I think you
> can catch the gist of what I'm doing, I hope :)
>
> (let [inst (i/instance "attribute_bitmaps")
> bmp (doto (RoaringBitmap.)
> (.add 9999999))
> srz (fn [it]
> (let [b (ByteArrayOutputStream.)]
> (with-open [o (DataOutputStream. b)]
> (.serialize it o))
> (.toByteArray b)))]
>
> (doto inst
> (i/clear)
> (i/put "test" bmp))
>
> (byte-streams/print-bytes (srz bmp))
> (byte-streams/print-bytes (srz (i/get inst "test"))))
>
> Here I'm just creating a bitmap and storing it in a cache. I'm then just
> printing the bytes of the thing I stored, as well as what it looks like
> coming back out of Ignite.
>
> I get the following warning in the logs:
>
> 10:16:45.730 [djura.local ~ nREPL-worker-3 ~
> o.a.i.internal.binary.BinaryContext] Class "org.roaringbitmap.RoaringBitmap"
> cannot be serialized using BinaryMarshaller because it either implements
> Externalizable interface or have writeObject/readObject methods.
> OptimizedMarshaller will be used instead and class instances will be
> deserialized on the server. Please ensure that all nodes have this class in
> classpath. To enable binary serialization either implement Binarylizable
> interface or set explicit serializer using
> BinaryTypeConfiguration.setSerializer() method.
>
> And really strangely, I get the same number of bytes back but some of them at
> the end have been zero'd out (first one is correct, second one went through
> Ignite):
>
> 3A 30 00 00 01 00 00 00 98 00 00 00 10 00 00 00 .0..............
> 7F 96 ..
> 3A 30 00 00 01 00 00 00 98 00 00 00 10 00 00 00 .0..............
> 00 00 ..
>
> The resulting object is still a valid RoaringBitmap, except all the values in
> the bitmap are wrong!
>
> I'm guessing from this and the logs, that OptimizedMarshaller is being used
> instead of BinaryMarshaller, and OptimizedMarshaller is not copying the
> internal fields of the class correctly.
>
> Would the recommended approach here be to create a custom class that extends
> RoaringBitmap and implements Binarylizable? I'm not sure if Binarylizable is
> a suitable approach for this situation where I don't control the source for
> the class in question. I have no knowledge of the internal fields of this
> class and really just want to ensure it survives the roundtrip through Ignite
> by using its own internal serialization mechanism.
>
> Luke.
>
