On Mon, 26 Apr 2010 01:29:07 +0900, Michel Fortin
<[email protected]> wrote:
Looks well done. There's one thing I'd suggest though. I'm pretty sure
you could make it even faster by skipping the mp_Object intermediary
representation and using templates. I know it's possible since I've done
it for a surprisingly similar serialization library I'm working on.
The trick is to reuse the same pattern in the unpacker as you're already
using in the packer. For instance, the packer has this function:
ref Packer pack(T)(in T value) if (is(Unqual!T == long))
so the unpacker could have this function (just changed 'in' by 'out'):
ref Unpacker unpack(T)(out T value) if (is(Unqual!T == long))
My library works by unserializing everything directly a the right place
in a data structure while it parses the stream. Looks like this:
MyStruct original;
Archiver archiver;
archiver.encode(original);
immutable(byte)[] data = archiver.outout;
MyStruct copy;
Unarchiver unarchiver;
unarchiver.input = data
unarchiver.decode(copy);
This is unlike mp_Object which is in itself an intermediary
representation that sits between the serialized data and the data
structure you actually want to rebuild. I still have something similar
to mp_Object as a convenience for types that prefer to implement a
custom unserialization process in an order not dictated by the input
stream, but this is less efficient:
void decode(ref KeyUnarchiver archive) {
archive.decode("var1", var1);
archive.decode("var2", var2);
}
Yeah, I know your approach(Protocol Buffers supports similar approach
using schema).
Current implementation purposes separating deserialization and conversion.
D has Tuple and Variadic Templates, so I can rewrite execute method
relatively easily if need.
What I'm trying to put to work now is a way to deal with multiple
references to the same object.
MessagePack doesn't consider multiple references to same object.
This support slows down serialization / deserialization and makes stream
deserializer difficult.
This problem poses little problem for product use.
I'd also like a nice way to deal with Variant, but I'm under the
impression this won't be possible without adding serialization support
directly into Variant, or into TypeInfo.
I agree this point. This problem might be solved if D has ADL(but I don't
like ADL).
Masahiro, sorry: this started as a useful commentary on your
unserializer's approach and I ended up instead promoting what I am
doing. Your library seems targeted at making a MessagePack serializer,
with an emphasis on having a simple and portable serialization format,
which is great when you want to communicate in this format.
Your understanding is correct. MessagePack is designed at efficient, small,
and fast serialization(thus not perfect). I think this simplicity meets
Phobos.
But on my
side, I care more about being able of recreating object graphs and
reinstantiating objects of the correct class when unserializing.
Perfect serialization needs environment support.
However, such serialization is very difficult if support.
(Cocoa, Java, and etc. Ruby's marshal is slow)
I don't know the library based perfect serialization.
That does not seem possible with your library, and MessagePack doesn't
support this so it doesn't seem likely it can be added easily, am I
right?
You are right.
The author of MessagePack considers supporting class and reference
extension.
But his conclusion is "There is no little merit in supporting".
