On 2011-08-20 18:16, dsimcha wrote:
This looks really great! I'll try it out sometime later. A few questions:
1. What other archiver formats besides XML might be useful? (I remember
binary can't work because of the way keys work, though messagepack is
probably a better option for lightweight serialization in general.
Orange's niche is portability and serializing as many types as possible.)
I assume any archive type that can unpack a value given a key will work.
I assume JSON and YAML will work as well. I no nothing about binary
serialization but I think messagepack can work.
Another problem might be pointers and slices. For example, at the first
step every array is assumed to be pointing to its own memory, i.e. not a
slice. After everything is serialized the serializer post process the
serialized data and replaces all arrays that points into another array
with a slice.
2. What are the prospects for submitting this for inclusion in Phobos?
Serialization is something basic and universally needed enough that it
should not require a third-party library.
From my part:
* Remove all Tango/D1 specific code
* Add documentation
* There are some few things left in the rewrite process
* I've made some modifications to the XML module in Phobos, these need
to be applied as well
* Probably something else I can't think of right now
I'm not particular happy about removing the Tango/D1 specific code if
I'm not sure that the library can be included in Phobos.
3. Given that all of the necessary introspection already works, what are
the prospects for creating a Clone archiver? A Clone archiver would not
actually serialize anything and instead would deep clone whatever data
structure it receives. If Orange gets into Phobos, cloning could be
recognized by std.concurrency as a safe, simple way to pass complex
object graphs between threads. I'm picturing an API something like this:
import std.stdio, whateverOrangeEndsUpBeingNamed.
struct Cloned(T) {
T obj;
}
Cloned!T cloned(T)(T obj) { return typeof(return)(obj); }
void spawmMe() {
receive((int[][]) { writeln("Received an int[][]."); });
}
void main() {
auto tid = spawn(&spawnMe);
auto mat = new int[][](42, 42);
// FAILS: Implicit sharing.
tid.send(mat);
// Works. mat is automatically deep cloned and the clone is sent
// to tid. send recognized the Cloned struct as special and
// processes the payload accordingly.
tid.send(cloned(mat));
}
Currently you could serialize the data an pass it send it over to
another thread and deserialize on the other side, but I'm guessing this
would be not very effective. I'm not exactly sure what's needed to be
done to make a clone archive and what can be shared but as you say, the
necessary introspection already work.
On 8/20/2011 11:13 AM, Jacob Carlborg wrote:
I've almost finished the rewrite of my serialization library Orange. I'm
hoping that someone wants to give it a try and see what issues/bugs are
found.
Project page: http://dsource.org/projects/orange
Source code: https://github.com/jacob-carlborg/orange
There are two usage examples on the project page. For more examples I
recommend looking at the unit tests in the "tests" directory.
Description:
Orange is a serialization library for D1 and D2, supporting both Tango
and Phobos. It can serialize most of the available types in D, including
third party types and can serialize through base class references. It
supports fully automatic serialization of all supported types and also
supports several ways to customize the serialization process. Orange has
a separate front end (the serializer) and back end (the archive) making
it possible for the user to create new archive types that can be used
with the existing serializer.
Features:
* Automatically serializes the base classes
* Supports events (before and after (de)serializing)
* Supports non-serialized fields and classes (you can say that some
fields in a class should not be serialized)
* Licensed under the Boost license
* Std/runtime library independent
* Extendable - possible to create new archive types and use them with
the existing serializer
* Serializes through base class references
* Serializes third party types
* Customization of the (de)serialization process, both intrusive and
non-intrusive
* Properly (de)serializes slices and pointers
Known Issues/Limitations:
* Due to limitations in the XML module provided by Phobos the XMLArchive
will only work with "char" as the template type with D2
* Due to several bugs/limitations in the compiler/runtime even the D2
version requires you to register the type when serializing through base
class references
* No built-in support for versioning
* Floating point numbers are not serialized as hexadecimal (D1)
--
/Jacob Carlborg
