Hi,
I was I little bit off this discussion for some time, and now being able to
look onto it with "fresh" view again. It appears to me now that there is
some confusion in an air. I must admit I was confused myself couple times
and gave invalid comments. Here is my current thoughts of that has happened,
happening and should happen.
As far as I understand Robert (sorry beforehand if I misinterpret
anything) starting this work to bring MFC (most influence) tradition of data
sterilization into persistent storage. The purpose was to make it portable,
more flexible, more powerful and convenient I do not know how it should be
named properly: sterilization library or persistence one, the idea is to
simplify traverse through the *existent* C++ structures for the purpose of
storing or restoring them. Later during review boosters came up with some
possible asymmetric usages of the framework, when one only would be
interested in specific output for example. Behind this we (seems) to forgot
the *key* word in original application area description. The nature of the
library tightly connected to some known at compile time C++ structures. This
should immediately rule out any application to the areas like XML parser,
that created meta structures dynamically. In general any form of meta data
could only define specifics of the format, like integer values compression
style, byte ordering (big/little Endean).
So now we could define to what kind of designs this library will be
applicable to and where not:
This library deals with C++ structures driven serialization
This library does *not* deal with stream-driven serialization
We could analyze the input data but in most cases only for validation
purposes. Another exclusion would be polymorphic pointers or size of dynamic
collections. But we always working in a strict frames of traversing
algorithm.
Now let me present some ideas about this library design. They may or may not
intersect with ones that already in an air but I will give it a try.
1. Traversing framework.
I saw started discussion about full fledged reflection library. But I do not
think we need to connect it with this discussion. While it's task that worth
persuading - serialization library itself only need some peaces of it, that
could be easily implemented. That's why I will call this component -
traversing framework. If you look into regular sterilization code you will
see that in most cases save/load looks very similar. I (as many others)
wanted to emphasize, separate and have this logic in one place. In fact
namely this traversing ability if serialization library was the only think
attractive to those who wanted to implement any kind of specific human
readable output (this format may not have load counter part at all and it
does not bother about a lot of other things)
Here where I introduce code like this:
//*******************************************************//
traverse.hpp
template<typename Traversee>
struct traverse_traits
{
template<typename Traverser>
static void traverse( Traverser& tr, Traversee& t )
{
tr.pre_traverse( t );
t.traverse( tr );
tr.post_traverse( t );
}
};
//__________________________________________________________________________
__//
template<typename Traverser,typename Traversee>
void
traverse( Traverser& tr, Traversee& t )
{
traverse_traits<Traversee>::traverse( tr, t );
}
//__________________________________________________________________________
__//
It's obvious that traverse_traits will need to be specialized for all
intrinsic types. Here one for int
template<>
struct traverse_traits<int>
{
template<typename Traverser>
static void traverse( Traverser& tr, int& t )
{
tr.traverse_intrinsic( t );
}
};
Next we will need to provide some specialization for STL constructs. Here we
will find some difficulties. Traversing dynamic collections is different
depend on what king of logic we are interested in or out. So I introduced
traverser category. Here an example how approximately it will work. Proper
implementation should work similar to one in collection_imp.hpp
struct traverser_category {};
struct in_traverser_category : traverser_category {};
struct out_traverser_category : traverser_category {};
//__________________________________________________________________________
__//
template<typename Traverser,typename Collection>
void
traverse_dynamic_collection( Traverser& tr, Collection& c,
in_traverser_category )
{
int size;
traverse_traits<int>::traverse( tr, size );
while( size-- > 0 ) {
typename Collection::value_type value;
::traverse( tr, value );
c.push_back( value );
}
}
//__________________________________________________________________________
__//
template<typename Traverser,typename Collection>
void
traverse_dynamic_collection( Traverser& tr, Collection& c,
out_traverser_category )
{
int size = c.size();
traverse_traits<int>::traverse( tr, size );
for( typename Collection::iterator it = c.begin();
it != c.end();
++it ) {
::traverse( tr, *it );
}
}
//__________________________________________________________________________
__//
#include <list>
template<typename T>
struct traverse_traits<std::list<T> >
{
template<typename Traverser>
static void traverse( Traverser& tr, std::list<T>& lst )
{
traverse_dynamic_collection( tr, lst, typename
Traverser::category() );
}
};
//__________________________________________________________________________
__//
For user convenience purposes we could introduce macro that will simplify
specialization for the UDT.
Here how would it look with one argument (real macro will need to use
BOOST_PP)
#define DEFINE_TRAVERSING( type, field_name ) \
template<> \
struct traverse_traits<type > \
{ \
template<typename Traverser> \
static void traverse( Traverser& tr, type& t ) \
{ \
tr.pre_traverse( t ); \
::traverse( tr, t.field_name ); \
tr.post_traverse( t ); \
} \
} \
Ok. That's it. It already pretty useful. As examples I implemented 2 trivial
traversers: trivial_xml_printer and csv_reader (one input and one output)
both using this framework for traversing. See attached files
2. Plain pointers management component
We need to separate logic responsible for tracking the pointer into
substitutable component. There several possible implementation of it. In
trivial case I would not track pointers at all cause I know that my
structures does not use them. In another cases I may use simplified form
that would not track repeating pointers, cause I know for sure that all my
pointers are unique. And finally one may want implementation that is
currently in the library that racks everything.
3. Polymorphic pointers management component
This component will be responsible for dealing with polymorphic pointers.
This one I would call the reflection policy. It will be responsible for type
identification, type dependency registry, and casting. It may be used by
second for it's work.
4. i|o_archive s
Archives should present implementation of input/output traversers (And
present more convenient interface that function traverse) The special care
in designing of this component should be taken about optional virtuality
(see my other post about how it could be implemented). We should try to
hardcode as little overhead as possible so that formats designers will have
more freedom.
5. Finally specific formats for each archive
It's questionable how would be the best way support user defined formats
through inheritance and template parameter. It's a bit too early to talk
about this.
Here how I envision the design of the Sterilization library.
Hope it has some useful grain.
Regards,
Gennadiy.
begin 666 traverse.hpp
M(VEF;F1E9B!44D%615)315](4% -"B-D969I;F4@5%)!5D524T5?2%!0#0H-
M"G1E;7!L871E/'1Y<&5N86UE(%1R879E<G-E93X-"G-T<G5C="!T<F%V97)S
M95]T<F%I=',-"GL-"B @("!T96UP;&%T93QT>7!E;F%M92!4<F%V97)S97(^
M#0H@(" @<W1A=&EC('9O:60@=')A=F5R<V4H(%1R879E<G-E<B8@='(L(%1R
M879E<G-E928@=" I#0H@(" @>PT*(" @(" @("!T<BYP<F5?=')A=F5R<V4H
M('0@*3L-"B @(" @(" @="YT<F%V97)S92@@='(@*3L-"B @(" @(" @='(N
M<&]S=%]T<F%V97)S92@@=" I.PT*(" @('T-"GT[#0H-"B\O7U]?7U]?7U]?
M7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?
M7U]?7U]?7U]?7U]?7U]?7U]?7U]?7R\O#0H-"G1E;7!L871E/'1Y<&5N86UE
M(%1R879E<G-E<BQT>7!E;F%M92!4<F%V97)S964^#0IV;VED#0IT<F%V97)S
M92@@5')A=F5R<V5R)B!T<BP@5')A=F5R<V5E)B!T("D-"GL-"B @("!T<F%V
M97)S95]T<F%I=',\5')A=F5R<V5E/CHZ=')A=F5R<V4H('1R+"!T("D[#0I]
M#0H-"B\O7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?
M7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7R\O#0H-"G1E
M;7!L871E/#X-"G-T<G5C="!T<F%V97)S95]T<F%I=',\:6YT/@T*>PT*(" @
M('1E;7!L871E/'1Y<&5N86UE(%1R879E<G-E<CX-"B @("!S=&%T:6,@=F]I
M9"!T<F%V97)S92@@5')A=F5R<V5R)B!T<BP@:6YT)B!T("D-"B @("![#0H@
M(" @(" @('1R+G1R879E<G-E7VEN=')I;G-I8R@@=" I.PT*(" @('T-"GT[
M#0H-"B\O7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?
M7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7R\O#0H-"G-T
M<G5C="!T<F%V97)S97)?8V%T96=O<GD@>WT[#0IS=')U8W0@:6Y?=')A=F5R
M<V5R7V-A=&5G;W)Y(#H@=')A=F5R<V5R7V-A=&5G;W)Y('M].PT*<W1R=6-T
M(&]U=%]T<F%V97)S97)?8V%T96=O<GD@.B!T<F%V97)S97)?8V%T96=O<GD@
M>WT[#0H-"B\O7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?
M7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7R\O#0H-
M"G1E;7!L871E/'1Y<&5N86UE(%1R879E<G-E<BQT>7!E;F%M92!#;VQL96-T
M:6]N/@T*=F]I9 T*=')A=F5R<V5?9'EN86UI8U]C;VQL96-T:6]N*"!4<F%V
M97)S97(F('1R+"!#;VQL96-T:6]N)B!C+"!I;E]T<F%V97)S97)?8V%T96=O
M<GD@*0T*>PT*(" @(&EN="!S:7IE.PT*(" @('1R879E<G-E7W1R86ET<SQI
M;G0^.CIT<F%V97)S92@@='(L('-I>F4@*3L-"@T*(" @('=H:6QE*"!S:7IE
M+2T@/B P("D@>PT*(" @(" @('1Y<&5N86UE($-O;&QE8W1I;VXZ.G9A;'5E
M7W1Y<&4@=F%L=64[#0H@(" @(" @.CIT<F%V97)S92@@='(L('9A;'5E("D[
M#0H@(" @(" @8RYP=7-H7V)A8VLH('9A;'5E("D[#0H@(" @?0T*?0T*#0HO
M+U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?
M7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U\O+PT*#0IT96UP;&%T
M93QT>7!E;F%M92!4<F%V97)S97(L='EP96YA;64@0V]L;&5C=&EO;CX-"G9O
M:60-"G1R879E<G-E7V1Y;F%M:6-?8V]L;&5C=&EO;B@@5')A=F5R<V5R)B!T
M<BP@0V]L;&5C=&EO;B8@8RP@;W5T7W1R879E<G-E<E]C871E9V]R>2 I#0I[
M#0H@(" @:6YT('-I>F4@/2!C+G-I>F4H*3L-"B @("!T<F%V97)S95]T<F%I
M=',\:6YT/CHZ=')A=F5R<V4H('1R+"!S:7IE("D[#0H-"B @("!F;W(H('1Y
M<&5N86UE($-O;&QE8W1I;VXZ.FET97)A=&]R(&ET(#T@8RYB96=I;[EMAIL PROTECTED]*
M(" @(" @(" @:70@(3T@8RYE;F0H*3L-"B @(" @(" @("LK:70@*2![#0H@
M(" @(" @.CIT<F%V97)S92@@='(L("II=" I.PT*(" @('T-"GT-"@T*+R]?
M7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?
M7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?+R\-"@T*(VEN8VQU9&4@
M/&QI<W0^#0H-"G1E;7!L871E/'1Y<&5N86UE(%0^#0IS=')U8W0@=')A=F5R
M<V5?=')A:71S/'-T9#HZ;&ES=#Q4/B ^#0I[#0H@(" @=&5M<&QA=&4\='EP
M96YA;64@5')A=F5R<V5R/@T*(" @('-T871I8R!V;VED('1R879E<G-E*"!4
M<F%V97)S97(F('1R+"!S=&0Z.FQI<W0\5#XF(&QS=" I#0H@(" @>PT*(" @
M(" @("!T<F%V97)S95]D>6YA;6EC7V-O;&QE8W1I;VXH('1R+"!L<W0L('1Y
M<&5N86UE(%1R879E<G-E<CHZ8V%T96=O<GDH*2 I.PT*(" @('T-"GT[#0H-
M"B\O7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?
M7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7U]?7R\O#0H-"B-D969I
M;F4@1$5&24Y%7U12059%4E-)3D<H('1Y<&4L(&9I96QD7VYA;64@*2!<#0IT
M96UP;&%T93P^(%P-"G-T<G5C="!T<F%V97)S95]T<F%I=',\='EP92 ^(%P-
M"GL@7 T*(" @('1E;7!L871E/'1Y<&5N86UE(%1R879E<G-E<CX@7 T*(" @
M('-T871I8R!V;VED('1R879E<G-E*"!4<F%V97)S97(F('1R+"!T>7!E)B!T
M("D@7 T*(" @('L@7 T*(" @(" @("!T<BYP<F5?=')A=F5R<V4H('0@*3L@
M7 T*(" @(" @(" Z.G1R879E<G-E*"!T<BP@="YF:65L9%]N86UE("D[(%P-
M"B @(" @(" @='(N<&]S=%]T<F%V97)S92@@=" I.R!<#0H@(" @?2!<#0I]
D(%P-"B\J*B\-"@T*(V5N9&EF("\O(%12059%4E-%7TA04 T*
`
end
begin 666 example1.cpp
M(VEN8VQU9&4@/&5X86UP;&4N:'!P/@T*(VEN8VQU9&4@/&EO<W1R96%M/@T*
M#0IS=')U8W0@=')I=FEA;%]X;6Q?<')I;G1E<@T*>PT*(" @('1Y<&5D968@
M;W5T7W1R879E<G-E<E]C871E9V]R>2!C871E9V]R>3L-"@T*(" @('1E;7!L
M871E/'1Y<&5N86UE(%0^#0H@(" @=F]I9"!P<F5?=')A=F5R<V4H(%0F('0@
M*0T*(" @('L-"B @(" @(" @<W1D.CIC;W5T(#P\("<\)R \/"!T+FED*"D@
M/#P@)SXG.PT*(" @('T-"@T*(" @('1E;7!L871E/'1Y<&5N86UE(%0^#0H@
M(" @=F]I9"!P;W-T7W1R879E<G-E*"!4)B!T("D-"B @("![#0H@(" @(" @
M('-T9#HZ8V]U=" \/" B/"\B(#P\('0N:60H*2 \/" G/B<[#0H@(" @?0T*
M#0H@(" @=&5M<&QA=&4\='EP96YA;64@5#X-"B @("!V;VED('1R879E<G-E
M7VEN=')I;G-I8R@@5"8@=" I#0H@(" @>PT*(" @(" @("!S=&0Z.F-O=70@
M/#P@=#L-"B @("!]#0I].PT*#0II;G0@;6%I;B@I#0I[#0H@(" @0R!C*#4I
M.R @(" -"B @("!T<FEV:6%L7WAM;%]P<FEN=&5R('1R.R @( T*#0H@(" @
7=')A=F5R<V4H('1R+"!C("D[#0I]#0H`
`
end
begin 666 example2.cpp
M(VEN8VQU9&4@/&5X86UP;&4N:'!P/@T*(VEN8VQU9&4@/&EO<W1R96%M/@T*
M#0IS=')U8W0@=')I=FEA;%]X;6Q?<')I;G1E<@T*>PT*(" @('1Y<&5D968@
M;W5T7W1R879E<G-E<E]C871E9V]R>2!C871E9V]R>3L-"@T*(" @('1E;7!L
M871E/'1Y<&5N86UE(%0^#0H@(" @=F]I9"!P<F5?=')A=F5R<V4H(%0F('0@
M*0T*(" @('L-"B @(" @(" @<W1D.CIC;W5T(#P\("<\)R \/"!T+FED*"D@
M/#P@)SXG.PT*(" @('T-"@T*(" @('1E;7!L871E/'1Y<&5N86UE(%0^#0H@
M(" @=F]I9"!P;W-T7W1R879E<G-E*"!4)B!T("D-"B @("![#0H@(" @(" @
M('-T9#HZ8V]U=" \/" B/"\B(#P\('0N:60H*2 \/" G/B<[#0H@(" @?0T*
M#0H@(" @=&5M<&QA=&4\='EP96YA;64@5#X-"B @("!V;VED('1R879E<G-E
M7VEN=')I;G-I8R@@5"8@=" I#0H@(" @>PT*(" @(" @("!S=&0Z.F-O=70@
M/#P@=#L-"B @("!]#0I].PT*#0IS=')U8W0@8W-V7W)E861E<@T*>PT*(" @
M('1Y<&5D968@:6Y?=')A=F5R<V5R7V-A=&5G;W)Y(&-A=&5G;W)Y.PT*#0H@
M(" @=&5M<&QA=&4\='EP96YA;64@5#X-"B @("!V;VED('!R95]T<F%V97)S
M92@@5"8@=" I#0H@(" @>PT*(" @('T-"@T*(" @('1E;7!L871E/'1Y<&5N
M86UE(%0^#0H@(" @=F]I9"!P;W-T7W1R879E<G-E*"!4)B!T("D-"B @("![
M(" @#0H@(" @?0T*#0H@(" @=&5M<&QA=&4\='EP96YA;64@5#X-"B @("!V
M;VED('1R879E<G-E7VEN=')I;G-I8R@@5"8@=" I#0H@(" @>PT*(" @(" @
M("!C:&%R(&,[#0H@(" @(" @(&1O('L@#0H@(" @(" @(" @("!S=&0Z.F-I
M;B ^/B!C.PT*(" @(" @("!]('=H:6QE*"!C(#T]("<@)R!\?"!C(#T]("<L
M)R I.PT*(" @(" @("!S=&0Z.F-I;BYU;F=E="@I.PT*#0H@(" @(" @('-T
M9#HZ8VEN(#X^('0[#0H@(" @?0T*?3L-"@T*:6YT(&UA:6XH*0T*>PT*(" @
M($,@8SL-"B @("!C<W9?<F5A9&5R('([#0H-"B @("!T<F%V97)S92@@<BP@
M8R I.PT*#0H@(" @=')I=FEA;%]X;6Q?<')I;G1E<B!T<CL-"@T*(" @('1R
5879E<G-E*"!T<BP@8R I.PT*?0T*
`
end
begin 666 example.hpp
M(VEF;F1E9B!%6$%-4$Q%7TA04 T*(V1E9FEN92!%6$%-4$Q%7TA04 T*#0HC
M:6YC;'5D92 \=')A=F5R<V4N:'!P/@T*#0HC:6YC;'5D92 \;&ES=#X-"B-I
M;F-L=61E(#QS=')I;F<^#0H-"G-T<G5C="!!('L@02AI;G0@9B ](# I.R!I
M;G0@(" @(" @(" @9FEE;&0[("!S=&0Z.G-T<FEN9R!I9"@I.R!].PT*<W1R
M=6-T($(@>R!"*&EN="!S(#T@,"D[('-T9#HZ;&ES=#Q!/B!V86QU97,[('-T
M9#HZ<W1R:6YG(&ED*"D[('T[#0IS=')U8W0@0R![($,H:6YT(',@/2 P*3L@
M0B @(" @(" @(" @('-T;W)E.R @<W1D.CIS=')I;F<@:60H*3L@?3L-"@T*
M03HZ02AI;G0@9BD@.B @9FEE;&0H(&8@*2![?0T*0CHZ0BAI;G0@<RD-"GL-
M"B @("!F;W(H(&EN="!I/3 [(&D\<SL@*RMI("D-"B @(" @('9A;'5E<RYP
M=7-H7V)A8VLH:2D[#0I]#0I#.CI#*&EN="!S*2 Z('-T;W)E*',I('M]#0H-
M"G-T9#HZ<W1R:6YG($$Z.FED*"D@>R!R971U<FX@(D$B.R!]#0IS=&0Z.G-T
M<FEN9R!".CII9"@I('L@<F5T=7)N(")"(CL@?0T*<W1D.CIS=')I;F<@0SHZ
M:60H*2